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ABSTRACT 

The primary purpose of the Symposium was to assist 
the veterinary medical educator in becoming a more effective and 
efficient teacher. The main focus was placed on the educational 
process and on newer ideas in education, with the expectation that 
application could be made to the totality of veterinary medical 
education. The Fifth Symposium was planned as a minicourse in design, 
presentation, and evaluation of learning experiences. The topics of 
minicourses, multi-media instruction, and mastery learning were 
emphasized. The meeting attempted to combine the features of a 
symposium (dining and free exchange), a conference (presentations by 
outstanding authorities in the field of education, media, and 
evaluation), and a workshop ^(individual participation and 
contribution) . The major presentations are published in this volume, 
and appendices provide a statement of the participant objectives, a 
list of the program contributors, and the results of a post-symposium 
questionnaire. (JR) 
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PREFACE 



The Fifth Symposium on Veterinary Medical 
Education was built on the excellent base provided 
by the previous four symposia. The First 
Symposium, held at Michigan State University, 
provided an overview of veterinary medical 
education. The Second, held at the University of 
Georgia, focused its attention primarily on clinical 
education. The Third* Symposium, again held at 
Michigan State University, dealt with 
p re-veterinary programs; while the Fourth 
Symposium, held at Texas A & M University, 
reviewed graduate education.-' After having 
considered the various components of our 
education program, it was appropriate that the 
Fifth Symposium focus on the educational process 
and newer ideas in education with the expectation 
that application could be made to the totality of 
veterinary medical education. 

The Fifth Symposium was sponsored by the 
American* Veterinary Medical Association in 
conjunction with the ^Association of American 
Veterinary Medical Colleges. It was initiated in 
1970 when the American Veterinary Medical 
Association's House of Delegates authorized its 
development. Partial funding was obtained from 
the National Institute of Health. The National 
Coordinating Committee consisted of 
Doctors R. E. Brown, University of^ Missiouri; 
J. Lieberman, Video Record Corporation; 
N. E. Hutton, Iowa State University; 
W. M. Decker, A.V.M.A. Office representing the 

A. V.M.A.; and Doctors C. F. Reed, Michigan State 
University; J. J; Stockton, Purdue University; and 
J.R.Welser, University of Georgia, representing 
the A.A.V.M.C. This committee developed the 
format, specified the general topics and selected 
the site for the Fifth Symposium. Following the 
selection of the University of Georgia as the 
location, a local planning committee composed of 
Doctors M. E. Adsit, L. M. Crawford, J. W. Harris, 

B. E. Hooper, R. E. Lewis, J. T. Mercer, 
D.E.Tyler, and J.R.Welser, chairman, was 
formed. The local committee further developed 



and implemented the plans for the Fifth 
Symposium. Invaluable assistance in carrying out 
the Fifth Symposium was received from the 
faculty, staff, and administration of the College of 
Veterinary Medicine, University of Georgia. 
Without their help and encouragement, the Fifth 
Symposium would not have become a reality. 

For the people who planned the Fifth Symposium, 
the response was gratifying in spite of Hurricane 
Agnes and a pilots' -st;rike. Thelegistered 
attendance of 325 wa*s more than double the. 
number ever to attend a symposium on veterinary 
medical education. In addition, the response of the 
participants in reading their pre-confererice 
materials, attending the general sessions, and 
participating in the workshops made the 
symposium a success. 

The Fifth Symposium was planned as a minicourse 
in the design, presentation, and evaluation of 
learning experiences. The topics of minicourses, 
multi-media instruction, and rriastery learning were 
featured. The meeting attempted to combine the 
features of a symposium (dining and free 
exchange); a conference (presentation by 
outstanding authorities in^the field of education, 
media, and evaluation); and a workshop 
(individual participation arid contribution). Based 
on this, it was nicknamed "Tricom" by some. 

The primary purpose of the Symposium was to 
i assist the veterinary medical educator in becoming 
'a more effective and efficient teacher. Specific 
^objectives were included in the program, and upon 
p re-registration' the participants were sent 
pre-conference reading materials. The participants 
were pre-tested'on the readings utijizing Trainer 
Tester cards donated by Van Valkenburgh, 
Nooger, and Neville. During the Symposium, each 
participant filled a notebook with copies of the 
formal presentations, commercial literature, 
worksheets, and copies of the produced 
minicourses. A post-symposium test based on 



pre-stated objectives was also administered to the 
participants utilizing the Trainer Tester cards. In 
addition, the participants also filled out 
pfe-symposiurn and post-symposium opinion 
questionnaires. A part of the post-symposium 
questionnaire has been included in the' Appendix. 

• 

The Symposium started with an overview of the 
challenges that face veterinary medical education 
in Dr. Luther Terry's and W. W. Armistead's 
presentations. Drs. Postlethwait's, Brown's, 
Allen's, and Mager's .presentations defined 
minicourses, pomted out their application, and 
provided the detailed instruction on how to make 
up a minicourse and write performance objectives. 
Following these presentations, the participants 
gathered in small vi/ork groups where they chose a 
minicourse topic and wrote performance 
objectives for their chosen topic. After appropriate 
critique and feedback, the particpants attended a 
presentation on media selection by Dr. Millard and 
visited media displays provided by veterinary 
medical' educators from all of the colleges of 
veterinary medicine. The participants were ihen 
provided a review of the field of instructional 
evaluation along with future trends by Dr. Hedges. 
After presentations directed at different methods 
of evaluations by Drs. Smith, Gardner, and Block, 
the participants met in their work groups to 
structure a pre-test and post-test for their 
minicourse. » 

The Wednesday morning session was devoted to a 
discussion of future trends in education and the 
educator's role in the affective domain. The 



presentations centered around the subjects of 
accountability (Or. Lessinger), career planning 
(Dr. Owens), and credits and degrees (Dr. Harris): 
Concluding the Symposium, T.H.Ham's 
presentation of "Prospective and Retrospective 
Views of Medical Education" stimulated the 
participants and provided ideas for the future. 

/"'jThe-^spin-off and follow-up to the Fifth 
^ Symposium has indeed been gratifying, with 
70 percent of the schools of^eterinary medicine 
holding post-symposium sessions' in the form of 
programs, discussion seminars, or circulating 
presentations along with summary statements. 
Three schools r^-created the Fifth Symposium for 
their faculties. In these cases, faculty summarized 
the speeches that were delivered at the Fifth 
Symposium, and individuals who attended served 
as discussion leaders. 

It is difficult to express apprisciation .to all who 
worked and contributed so much to .'the Fifth 
Symposium; however, without the support of the 
. faculty and the u-ntiring work of theHpcal 
committee, it would never have been as successful. 
Ultimately the credit goes to the participants who 
came, were enthusiastic, and worked "hard at the 
Symposium. Hopefully the effect of the Fifth 
Symposium will not end with the publication of 
these proceedings. 



J. R.Welser 

Editor 

r 
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WELCOME REMARKS 



FIFTH SYMPOSIUM ON VETERINARY MEDICAL EDUCATION • . 

R. B, Talbot 



Welcome: 



It is certainly my pleasure to welcome all of you as 
participants in this Conference on Veterinary 
Medical Education. 

The theme of this conference is one I am vitally 
interested in and I want to congratulate you on 
choosing to participate in it. 

Let me for a moment then tell you as words of 
Vi^elcome a few things about the University and the - 
College of Veterinary^Medicine. 

1. Georgia was the first state to charter a state 
supported university. In 1784, the General 
Assembly set aside 40,000 acres of land to 
endow a college and in 1785, the University 
of Georgia was incorporated by an act of the 
General Assembly. 

2. The University was actually established in 
1801, when a committee selected the 
present land site. The first class was 
graduated in 1804. 

3. The curriculum of traditional - classical 
studies was broadened in 1843, to include 
courses in law and again in 1872, when 
federal funds became available for 
agriculture courses. ' 

4. Today the University is composed of 
13 schools and colleges plus several auxiliary 
divisions such as the Center for Continuing 
Education, in which vve are meeting today. 

5. In 1931, the General Assembly of the state 
placed all state supported institutions of 



higher education, including the University, 
in one organization known as the University 
System of Georgia. The System is governed 
by a single Board of Regents with one 
executive officer, the Chancellor. 

6, The University is a dynamic multifacited 
• institution today fulfilling instructional, 
research, and service responsibilities. There 
are approximately 18,000 full-time resident 
students enrolled and 1,900 faculty 
members. . ' 



The present College of Veterinary Medicine was 
organized in 1946, although it was preceded at the 
University by a degree program, in Veterinary 
Medicine from 1918 to 1933. 

The College has experienced extensive growth and 
development since 1964. During this period, the 
faculty has grown from 37 to 101 while the 
professional student enrollment has been held 
fairly constant or around 250. Graduate student 
enrollment, however, has increased from less than 
1 0 to more than 50 graduate students. The College 
is composed today of seven departments plus two 
diagnostic laboratories, one in Athens and one at 
Tifton. 

We are extremely proud of our faculty and student 
body at the College and invite you to spend as 
much time as possible with them during your stay 
here. 

It is my pleasure to again welcome you to the 
campus. If I can be of any personal assistance 
while you are here, I hope you will not hesitate to 
ask. 
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NEW HORIZONS FOR VETERINARY MEDICINE 

Luther L. Terry, M.D. 
Vice President for Medical Affairs, University of Pennsylvania, Philadelphia 



As I discuss the report of the Committee on 
Veterinary Medical Research and Education with 
you, I am reminded of the story of two illiterate 
country boys neither of whom had learned to tell 
time. One day one of the boys showed up with a 
bulge in the watch pocket of his trousers and a 
large chain emerging from the pocket and 
extending into the side pocket. The second young 
man on observing these facts and realizing that his 
friend was unable to tell time inquired, "What 
have you got on that chain in your pocket?" To 
which his friend responded, "It is a watch, want to 
see it?" With that, he withdrew the watch from his 
pocket and displayed it prominently. The second 
fellow responded, "Un-huh! What time is it?" The 
first pushed the watch toward his friend and said, 
"There it is." The second boy peered at it for a 
long moment and replied, "Damned if it ain't!" 

In referring to our committee's report I am prone 
to push it at you and say simple, "There it is." 
However, for fear that you might respond like the 
second boy, I feel compelled to point out some of 
the background of the committee and remark 
about some of the contents of the report. 

For many years, I have had a deep professional 
interest in veterinary medicine. This interest was 
kindled in part from my many years on the staff 
of the National Institutes of Health. During this 
period the importance of laboratory animals in 
biomedical research was indelibly impressed upon 
me and the applicability of animal models of 
importance to manl^ind was equally impressive. 
Furthermore, the importance of having specialists 
in animal medicine as a part of the research team 
was equally evident. 



Then after I became Surgeon General, for the first 
time in the nation's history we were beginningito 
convince the Congress of the need for federal 
assistance in the education and training of our 
health professionals. This culminated in 1963 with 
the Congress first enacting the Health Professions 
Educational Assistant Act. During the planning of 
this proposed legislation it was necessary for me, 
the members of my staff, and our consultants to 
define quite clearly the various categories of the 
health professions in order that we might make 
valid recommendations to the Congress. As one 
analyzed the situation it became quite clear that 
veterinary medicine was an essential component of 
our entire biomedical system from the standpoint 
not only of research, but also in the understanding 
of many human diseases and the prevention and 
management of those diseases. 



In 1965, when I left the Public Health Service and 
became Vice President for Medical Affairs at the 
University of Pennsylvania, I discovered that I 
inherited a degree of administrative responsibility 
for a School of Veterinary Medicine as well as 
School of Medicine, Dental. Medicine, Nursing, and 
Allied Medical Professions. It was at this time that 
I first had the opportunity to observe closely the 
education and training of veterinarians and to 
learn in great depth of the important biomedical 
research being performed in our veterinary medical 
schools. I can assure you that I was not 
disappointed; I was elated to see the contributions 
of^ veterinary medicine to human health and 
welfare. I became a complete convert to the 
importance of veterinary medicine to human 
health. 
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It was in the light of the above experience that I 
was delighted in 1969 when the officials of the 
National Academy of Sciences — National 
Research Council invited me to chair a committee 
to study the subject and to make 
recommendations for national guidance in this 
regard. Furthermore, it was a great assurance to 
me when I was requested to assist in the selection 
of the members of the committee. We spent 
several weeks reviewing the background of 
potential members and in selecting persons who 
were not only highly qualified, but who would 
truly represent the various aspects of veterinary 
medicine in the field of biomedical research and 
education. Frankly, I think that I am justified in 
bragging about the committee because I think we 
obtained the services of a group of outstanding 
persons who were willing to dedicate a lot of their 
personal time to the subject. Each member had a 
full-time responsibility to his home institution, but r 
each of them was willing to give of his personal 
time, and many long weekends and holidays were 
devoted to the study. I am grateful for the x 
contributions of each and every member of the 
committee. 

In speaking of the background of this report I 
should not fail to mention that the Academy was 
stimulated to establish this study from a request 
supported by a grant from the ' American 
Veterinary Medical Association. You will note in 
the report that other organizations, both 
government and private, also supported the 
project. During the entire course of our studies, we 
received strong support from the AVMA, the 
Association of Veterinvy Medical Schools, the 
Department of Agriculture, the Public Health 
Service, industry, and many other organizations 
and individuals. The entire staff of the Academy 
was cooperative, but we owe special thanks to our 
full-time staff member. Dr. Samuel Abramson, a 
veterinarian, and to Dr. Russell Stevens, Secretary 
of the Division of Biology and Agriculture, NAS. 
It is impossible to recognize all of our sources of 
information and advice, but the committee is 
grateful to each and all of them. 

Now that Newi Horizons for Veterinary Medicine 
has been completed and published, I am concerned 
for the first time. This concern reminds me of the 
age-old story of the apartment dweller who, when 



preparing for bed, heard clearly his upstair^ 
neighbor drop his first shoe on the floor and was 
unable to sleep for the entire night while awaiting 
the impact of the second shoe being dropped. Dur 
report has been "dropped" and I am greatly 
concerned that I have heard little response about 
its contents. I sincerely hope that today the 
second shoe will be "dropped" and that we will 
have* a free and open discussion of the report. 
There must be persons here who have "good" and 
"bad" things to say about it. I am further 
appreciative that other members of our committee 
are present to hear and to share with me in this 
critical discussion. 

I would like to select certain general 
recommendations from the report and also to 
mention some of the more specific details: 

1. Veterinary medical schools have historically 
grown-up in the atmosphere of land-grant 

^ colleges and the result has been a much 
closer relationship to agriculture than to 
human medicine. Dver the years veterinary 
medicine has undergone a series of 
transitions ranging from services almost 
entirely limited to diagnosis and treatment 
of animal diseases during the early days, to 
the more recent encouragement of • 
veterinarians in an array of medical services 
relating to agricultural 'enterprises and food 
production, industry, companion animals, 
the public health, research and teaching. In 
considering this* evolution it is quite clear 
that veterinary medicine should not release / 
its ties to agriculture, but the profession is/ 
clearly, merging closer to human^medieitle ' 
and this relationship should be strengthened. 
It is my opinion that a veterinary school 
which is geographically near and/or has a 
working relationship to a university medical 
center is fortunate. Obviously, it is- not 
feasible for most of our veterinary schools 
located away from medical centers to be 
moved. But it is possible for those schools to 
associate themselves more directly with 
human medicine in their vicinity. 
Furthermore, the importance of this shared 
relationship between human and veterinary 
medicine should be borne in mind by those 
persons who have a responsible role in the 
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location of new schools. It is a certainty, in 
my mind, that those two interrelated 
professions will need to work more closely 
in the future. 

Just as in human medicine, there is the need 
in veterinary medicine for major revision in 
our core curriculum and for up-dating many 
of our teaching methods and techniques. 
Increasing specialization! in veterinary 
medicine will necessitate some broadening of 
the core curriculum, but it also makes it 
mandatory that the veterinary student be 
given an opportunity for special and 
intensive experier^ce in a particular field 
while still in undergraduate professional 
school. Students entering professional 
schools today are far better prepared than 
comparable students of previous years and it 
is an opportunity and challenge^ for our 
educators of today to take advar)tay[e of this 
great asset. It is our responsibility to not 
force them into the monotony of repeating 
subject matter but to use this additional 
time to introduce them to new learning 
experiences and to give the student an 
opportunity for special attention to subjects 
of special individual interest to him. 

I shall not go into a discussion of many of 
the details of veterinary medical education 
considered in the report. But I wish to call 
your attention additionally to improvement 
in the use of teaching methodology to 
supplement the time and ability of the good 
teacher. Self-teaching devices have a real 
place in professional education, not only to 
supplement in*school learning, but to give 
the trainee an experience which can be 
valuable to his continuing education 
throughout life. ' 

Veterinary medicine has lagged far behind its 
sister profession in the development of more 
and better formal post-graduate programs to 
prepare its members for practice or research. 
It is startling to realize that with 
1240 graduates of our veterinary schools in 
1970-71 there were less than lOOformal 
internships available for their post-graduate 



training. An even worse condition prevails in 
relation to residency training leading to 
exceptional proficiency in the clinical 
sciences. Since this type of training is less 
formalized and standardized, it is difficult to 
give an exact estimate of the number of 
residency training spots available but in ; 
1969 there were only seven j^fetertnary;;^^^ 
schools offering residency training programs 
and there were only 41 residents in training 
that year. 

Of course, it is recognized that not all 
post-graduate training of veterinarians is in 
formal intern or residence training programs 
and that these experiences are primarily 
designed for training to practice. There are 
advanced degree programs leading to a 
masters or doctors degree in all but one of . 
our veterinary schools. During the 1.969-70 
acadefnic year, 312 veterinarians were 
enrolled in Ph.D* programs and 321 were in 
masters degree programs. In addition, 
veterinarians get advanced degree training in 
departments of veterinary science, medical 
•schools, and basic sciences departments of 
universities. Non-degree training is available 
in many government laboratories arid at ; 
private research laboratories, but these 
programs are commonly informal in content 
and are frequently tailored to the service 
needs of the particular organization. 

To my mind, one of the most glaring 
deficiencies in our veterinary educational 
and training activities is in the area of 
post-graduate training. We need more and 
better internships, residency, post-graduate 
degree, non-degree, and specialty training 
programs. This matter is so urgent that all of 
our institutions with responsibilities in this 
area should address themselves to this great 
need. Furthermore, there should be a 
combined effort in this direction which 
should be led by the AAVMC and the 
AVMA. Uniformity and acceptable 
standards should be of paramount interest. 

4. Professionals generally are becoming 
increasingly aware of the need to continue 



their education beyond the period devoted 
to earning an academic degree. The rapid 
pace of scientific advance demands that 
a professional person pursue a program of 
"continuing education" if he is to be 
proficient in his profession. Since 1962 the 
AVMA has given leadership in stimulating 
the development of courses for this purpose 
and has provided information on the 
availability and content of specific programs. 
This is a step in the right direction but there 
is mucjTjnore to be done. Therejhould be 
more*^ collaboration and discussion of 
responsibility between the colleges of 
veterinary medicine and the professional 
associations to facilitate the evaluation of 
more effective, expanded programs. Our 
committee has suggested how these 
objectives might be obtained. Jn addition, 
the committee has specifically recommended 
that the AVMA Council on Education 
should extend its efforts as follows: 

"A program to structure appropriate 
continuing education courses should 
be developed. Certain lectures and 
demonstrations serve best, perhaps, 
without extensive involvement of a 
national association, but many courses 
can be selected and organized for 
sequential treatment or directed 
toward specific goals on a national, 
integrated, and coordinated basis. 

"A system of evaluation of the quality 
- and effect of continuing education 
programs should be developed." 

I commend this advice for your 
consideration and action. 

5. One of the committee's very prominent 
concerns was in relation to the training and 
utilization of paramedical personnel. As in 
human medicine, /there is a growing 
awareness of the deficiencies in this regard 
and an appreciation that this is one of the 
best ways by which we may more rapidly 
and effectively increase the availability of 
quality veterinary medical services. 



When one reviews the available information 
on the subject, it is startling to learn of the 
very low technical-to-professional staff ratio 
in our veterinary schools. And yet, here is 
the greatest opportunity, not only to train 
paramedical personnel, but also to "extend 
the arms" of our teaching staff and train the 
young professional to use such personnel 
effectively. For example, it has been clearly 
shown in dentistry that the most opportune 
time to train a professional in the use of 
auxiliaries is during his formal education, 
both undergraduate and immediately 
thereafter. Furthermore, there are no areas 
related to veterinary medicine which are 
more qualified to train paramedical 
personnel than our professional schools. A 
veterinary school's involvement in such 
activities need not be confined to its 
immediate premises but can be effectively 
extended by affiliated programs with private 
practice groups and other institutions 
utilizing veterinary medical services. 

The committee has pointed out a particular 
area in which technicians could be used 
more effectively and thus permit the 
veterinarian more time for responsibilities 
which are truly of a professional 
nature — lay meat inspectors. It has pointed 
out that at least one state (New York) 
requires that lay meat inspectors have an 
associate degree after two years of junior 
college training in a food, meat, and dairy 
technology program. The committee felt 
that this effort deserved commendation, but 
also recognized that the effectiveness of such 
inspectors depended upon the quality of the 
training program plus the adequacy of 
professional supervision thereafter. Junior 
colleges must have professional participation 
in such teaching if it is to be meaningful and 
effective.^ 

General medical technologists can be trained 
to work as a part of a veterinarian*led team 
in much the same way as when serving as a 
part of a team in human medicine. Because 
the role of animal technicians is still 
evolving, it was not felt that any fixed 
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curriculum or pjriod of training can be 
recommended at this time. However, 
graduates of college-level programs should be 
encouraged to organize themselves so that 
they can develop a recognized certifying and 
accrediting program and, ultimately, 
standardize course curricula with established 
minimal training requirements. 

There are many other specific 
recommendations of the committee, but in 
closing I would like to call your attention to 
one other area to which considerable 
attention was devoted but with which it was 
not content with the information which 
evolved. I am referring to the role of 
companion animals in contributing to 
human health and welfare from the 
emotional standpoint. I am not referring to 
anything as utilitarian as a riding horse, 
hunting dog, or a seeing-eye dog. 

There is a growing belief among psychiatrists 
and psychologists that companion animals 
fulfill important emotional needs and hence 
contribute significantly to human well-being. 
All of us have been impressed on repeated 
occasions by the importance of a pet to the 
elderly, the lonely, and to the mentally ill or 
mentally retarded person. The tragedy of the 
illness or death of such a pet can be literally 
devastating to many such individuals. Yet, 
the role played by these animals has not 
been adequately studied. Consequently, we 
sometimes find ourselves in an 
uncomfortable position to respond when a 
legislator complains, "Why should public 
funds be used to help train a person who is 
to become a pet doctor?" I submit to you 
that such an attitude is not justified, and we 
as a nation are devoting an increasing 
amount of professional veterinary resources 



to the care of household pets — a trend 
which is likely to continue. 1 urge the 
members of the veterinary and medical 
professions to pay more attention to this 
subject and to institute collaborative studies 
which will .give us more objective 
information. Unless we can get more reliable 
information about the importance of pets to 
the mental health of ^people and to 
psychotherapy, a serious threat will continue 
to hang over the heads of those of us who 
are proponents of public assistance to 
veterinary research and education similar to 
that of hurran medicine. 



Pro^bly the. most controversial issue dealt with in 
our report is the number of veterinarians needed in 
the future in various categories of practice, as well 
as, the total needs for all types of practice. I am 
sure that some may disagree with our conclusions, 
but I would point out that the committee devoted 
a great deal of attention to this subject. Every 
known source which might be helpful was 
consulted. Furthermore, in Appendix H we have 
described in detail how we arrived at those 
conclusions. I commend your attention to this 
section for it is a matter of vital interest not only 
to veterinarians but to the entire nation. 
Unfottunately, our most optimistic predictions 
will result in a significant deficiency in the number 
of veterinarians by 1980. This shortage can be 
partially atoned for by the development of more 
effective utilization of the existing personnel, but 
we must continue to insist upon greater support 
for veterinary medical research and education at 
both the state and national level. 

All of us who served on the committee hope and 
trust that our efforts will give leadership for 
advancement in the entire field of veterinary 
medicine. 



DISCUSSION 
The Committee *pf the National Academy of Sciences 
^'Report on Veterinary Medical Research and Education" 

Panel 

L, L. Terry - 
W< H< Eyestone* 



Moderator 
R. B. Talbot 



Q. Was the effect of a decrease in the 
population explosion on the projected need 
ifor Veterinarians considered? 

A. (Dr. Eyestone): Yes, that point was 
considered very seriously. The population 
reduction projected for the next ten years 
might reduce the need for professional 
personnel. As Dr. Terry noted in his report, 
some professionals are being released from 
... the lay-type activities in meat inspection by 
lay help. In a practice situation we could 
probably reduce the need for professional 
people with lay help. 

(Dr. Terry)- Most of you are aware of the 
change that is going on in human medicine 
today in terms of health manpower 
personnel in the various categories. One of 
the very popular modes at the moment is the 
physician assistant, a person less well-trained 
than a physician to assist in the delivery of 
. medical care. Some of these programs 
encompass a few months training before the 



person is put in the field under the 
supervision of a physician to others which 
area a full four-year baccalaureate program. 
Frankly, human medicine is a confused mess 
in this regard. Part of the confusion is due to 
the realization that many things that 
physicians are doing could be performed by 
trained technical supportive personnel. If we 
are going to get better distribution of 
medical care,, we are going to have to utilize 
.every aspect in* trying to support the 
physician and extend his expertise to cover 
more areas. I think the'same situation is 
facing veterinary medicine today, though it 
has not quite crystallize±or precipitated yet. 
This is one of the areas in which veterinary 
medical schools need to give clearer and 
stronger leadership in order that the type of 
personnel, their proper training, and their 
utilization under professional supervision be 
defined. 

On what basis was the additional need for 
Veterinarians projected since they vary from 
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the projected need as published by the 
Bureau of Health Manpower? 

A. (Dr. Eyestone): The estimates from the 
Health Manpower Act are extrapolations of 
growth in the past, whereas the estimates in 
this report were based more on professional 
estimates of what might happen to certain 
segments of the profession and adding it all 
together to come up with a number. History 
has shown that the increase in the numbers 
of veterinarians and changes in th^ 
activities doss relate to population growth 
even though their distribution does not 
relate to centers of population. Veterinarians 
still largely relate to the concentration of 
food animals in the country. Even though 
we tend to follow population growth, there 
are other factors to consider. We don't 
totally relate to population growth.. New 
areas are becoming involved, 
environments, etc. that Veterinarians are 
becoming more and more involved and 
interested in. There has been a reduction in 
the need for veterinarians in the food animal 
area by virtue of the mass production 
methods that are being used. This has been 
during a population growth era and from 
what we've discovered does not lead to a 
drop in the need for veterinarians. Lack of 
population growth doesn't mean necessarily 
that we've stopped growing. Contrary to the 
public opinion, we will still have a gradual 
increase in the population. I believe the 
figures which you see in this report are 
related to our expected population growth, 
not our growth due to other areas. 

Q. 'vj/here do you feel lay help or. Veterinary 
technicians should be trained? 

A. (Dr. Eyestone): Their training should be 
related largely to the training in professional 
schools where professional students would 
have the opportunity to observe their use 
and training close at hand, where students 
can become accustomed to using auxiliaries. 
Also, this is the best place for this type of 
training. In the manpower law it states this 



must be the place of their training in the 
case of medicine. 

(Dr. Terry): I have mentioned in my 
remarks that certainly not all of this training 
must be, restricted to veterinary medical 
schools. On the other hand, I do feel that 
our veterinary medical schools are the ideal 
locus for continuing development and- 
improvement. In this regard, I think there is 
great incentive both to the trainer and to the 
trainee if carried out properly. In referring 
to the primary and important load of the 
veterinary school in the training of technical 
people, I didn't mean to infer at all that this 
should be the only point of training. In the ^ 
first place, I think standards and programs 
for training of these people should be 
developed. In some veterinary schools, 
faculty may be decreased by 
10 or 20 percent if scuff work jobs were left 
to technical help rather than to professional 
persons who - shouldn't be doing them 
anyway. The same thing is true in private 
practice. Whether 5, 10, or 20 percent or 
whatever it is, there is considerable segment 
of work performed by a veterinarian, both 
teachers in our school as well as practicing 
veterinarians in the field, which should not 
be done by professional people. It is up to us 
to divise methods and programs for the 
training of people to do these jobs. Serving 
as a technician is a perfectly honorable 
profession, but at the same time does not , 
require the training that is required of a 
professional person such as a veterinarian. I 
would urge that much more be done in this 
direction and we cannot only improve the 
availability of veterinary medical care but at 
the same time we can do a lot better job of 
medical practice. 

Many groups and individual veterinarians 
will be training technicians for particular 
roles in their own office or practice area. 
This is to be expected. At the same time I 
would point out to you that many 
well-trained technical people may be picked 
up from the human medical 
field: radiological technicians and 
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laboratory technicians and other people who 
are trained in our hospital and medical 
centers and which-can be valuable additions 
to veterinary practice and training. . 



Q. What type of quality control is beirig placed 
on technical training schools? 

A. (Dr. Eyestone): In some cases the state 
veterinary association and the college are 
doing some supervision over some of the 
activities involved in the Associate of Arts 
degree. Certainly the junior colleges', and 
other commercial training programs that 
have sprung up have been having problems in 
terms of agreement on what these people 
really ought to have . as training and 
supervision of graduates. And they will have 
problems until we get to the point of state 
licensure. I don't , see how we can fully 
control the situation. However, it seems to 
me that in the state boards of veterinary 
medical examiners and in states where a 
veterinary college exists there should be 
reasonably close supervision over the kinds 
of curricula, teaching, and preparations for 
an associate of arts degree. Certainly there 
are types of training that can be done nicely 
in such a setting. But when one considers a 
veterinary technician, by definition, 
analogous to the physician assistant, that 
kind of person really ought to be trained in a 
veterinary college clinical setting. 

Q. I can cite seve^al.,examples where graduate 
•technician trainirig programs are- assuming 
professional duties. How can this^e^ 
prevented? 

A. (Dr. Terry): -1* fear, I literally fear, some of 
the technical and semi-professional people 
that are being turned out by some of these 
schools without acequate supervision, 
professional leadership, training, guidance, 
and instruction. To my mind the biomedical 
area must do everything it can to assist and 
stimulate the development of adequate 
training programs so that we will not have 



turned out an albatross to hang around 
someone's neck. I think we're probably in 
for some rough times in this regard, more so 
in the human medicine technical area than in 
veterinary medicine. 

Practically every. community in thfs country 
wants a community college. One of the big 
fields in many of the comrnunity colleges is 
not the liberal " arts associate degree but 
subprofessional technical training, especially 
in the biomedical area. I think all of us are 
considerably concerned about the 
availability of quality teachers, guidance, 
and instruction for the people who are being 
trained in these areas. I think it's important 
for veterinary medicine to get its foot in the 
door so to speak, and to exert some 
leadership in helping to establish the nature, 
quality, and minimal requirements of the 
'training of the subprofessional technical 
people which will serve us and society. 



Q. In a recent report of the Pan American 
Health Organization, it was recommended 
that the technical assistants to veterinarians 
be called animal health assistants and their 
training be carried out by colleges of 
agriculture. Can you comment? 

A. (Dr. Terry): I'm wondering if, as Linus in 
the comic strip, veterinary medicine is losing 
its security blanket. In talking about the 
training of technical, paramedical, and 
subprofessional personnel whfch are to be 
used by veterinary medicine: if the 
veterinarian is afraid to have his name 
associated with them and wants to call them 
animal health care assistants rather than 
veterinary assistants, and furthermore, if he 
doesn't allow them to put a foot in his 
school, I fail to see the point! To me, 
clearly, the veterinary medical profession 
must give leadership in this direction and I 
would hope that people here are in the best 
position to give that leadership. I'm also 
concerned as to whether the veterinary 
medical profession generally, and our 
veterinary medical schools specifically, are 
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willing to step into this area and give some 
real meaningful leadership. 

(Dr. Eyestone): I think we gave a great deal 
of consideration to this point of view and 
attempted to be totally objective about 
measuring anything but effectiveness of the 
personnel involved. The only place to do this 
kind of training was in, around or under the 
supervision of colleges of veterinary 
medicine; whether you call them veterinary 
technicians, or what have you. Although it 
seems to me these comments are more often 
generated by the practitioners in our group 
than by those of us who are not in practice, 
people who had used well trained assistants 
in their practices seemed to firmly agree 
about the issues that we have mentioned and 
that these assistants should be trained by 
veterinary colleges. As Dean Tietz 
mentioned, perhaps this could be done at 
some school in Denver, Colorado under the 
supervision of the College of Veterinary 
Medicine, with the students eventually 
coming to the clinic at the College of 
Veterinary Medicine for final training before 
being released to practitioners. As is strongly 
reflected in the report, not only would the 
technicians who evolved from this 
program be working in the proper 
environment, but the senior students would "^ 
learn how to use them in their future 
practice. 



Q. Will the participant pay for continuing 
education, or can we look for public funds 
to provide re-training and continuing 
education programs? 

A. (Dr. Terry)-: Probably the best answer that I 
could give would be based on our experience 
in human medicine in continuing education 
in medical schools and the role of public 
funds in relation to this. 1 do not feel we 
. have continuing education as well organized 
■ in human medicine as we should have, 
although in recent years we have developed 
in terms of quality and areas covered. 
Generally speaking, public funds have been 



made available for continuing education in 
human medical areas only to institutions and 
not to support the expenses of the person 
attending. In addition, the support to the 
institutions has not usually been enough to 
avoid them requiring tuition. There is a 
general tedency on the part of our legislators 
to feel that in . contrast to a student who is 
going through school, a practitioner, 
whether he is working for an industrial firm, 
the Federal Government, or a private 
practice, is an earner. Therefore, he should 
pay at least in part, if not totally, for his 
upkeep, refurbishment, and training in his 
chosen field of continuing education. Thus, I 
imagine that some of our schools will be 
given assitance in a limited form from public 
funds but, in general, I think that most of 
these programs will need to be 
conceptualized on the basis that participants 
will pay a large part of the cost of 
instruction. Universities and other 
institutions are obviously going to 
underwrite some of the basic expenses, and 
professional organizations like the A.V.M.A. 
should also contribute. However, a large part 
of it will have to be carried by the individual 
professional who is getting the continuing 
education and, after all, it is tax deductible. 

Q. What is the role and importance of 
companion animals, in our society and do we 
have data to support their importance? 

A. (Dr. Terry): The committee was greatly 
disappointed by the amount of really 
objective information from any source on 
the subject. At the same time everyone of us 
as individuals, whether we are proficient 
veterinarians, medical scientists, or other 
professionals working in the area, are 
convinced of the great importance of 
companion animals, particularly pets in 
•relation to human health and welfare. I 
think we need much more study and 
documentation, of well conceived studies 
that are done by capable psychiatrists, 
psychologists, animal behaviorists, and 
veterinarians. These studies should be done 
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in a collaborative way, so that much more 
evidence can be derived in this area. I think 
that any ph/sician who has taken care of 
patients such as the elderly, the lonely, the 
mentally ill, or mentally retarded, cannot 
but help having recognized the importance 
animals and pets play in the lives of these 
people. But we just do not have enough 
documented information. Time and time 
again I have been clobbered by some 
legislator" who says, "Why should public 
funds be used to educate somebody who is 
going to become a pet doctor?/' which to 



my mind is a completely idntic statement 
and unappreciative of the importance of pets 
in relation to human life and welfare. Yet 
somehow or another we have to obtain more 
and better information on the subject in 
order to overcome some of these prejudices. 

(Dr. Talbot): Our time precludes any more 
questions. I am sure Dr Terry and Cr. Eyestone 
would be glad to answer any additionarinqu'ries 
on an individual basis. I would like to thank 
Drs. Terry and Eyestone for coming and adding so 
much to our Symposium. 
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THE WINDS OF CHANGE 

W. Armistead, D.V.M. 
Dean, College of Veterinary Medicine, Michigan Stale University, East Lansing 



Introduction 

Let me begin by explaining that the title, "The 
Winds of Change," was assigned to me; I did not 
select it. But I happily accepted it because it 
seemed so appropriate. When one thinks of a dean, 
one just naturally thinks of wind. 

Seriously, the title is appropriate to the theme of 
the symposium and ! am honored to have been 
asked to make this contribution to the program. 
As I understand it, my assignment is to interpret 
what changes are "in the wind" for veterinary 
medical education. It is an important assignment 
and one which I do not feel unusually qualified to 
carry out. But 1 shall give it a try, with confidence 
thai what I overlook will be filled in by others at 
this meeting. 

That there are significant changes in the wind is 
indisputable. The real questions are (1) the nature 
of these changes and (2) their implications for 
veterinary medical education in the decade ahead. 

In discussing these matters I shall try merely to 
help set the mood and stage for this symposium. If 
by chance I should have any impact beyond that, I 
shall be surprised but grateful! 



What Is in the Wind? 

Veterinary medical education in the next decade 
will be affected profoundly by many of the issues 
impinging upon higher education as a whole. Of 
these the most important probably is funding. 
Many of us in veterinary medical education have 
not yet seen — or at least have not yet 



accepted — the full implication of national trends 
in financing higher education. 

One thing is certain. We shall not see again for a 
while a period of sustained growth such as we 
experienced in the 1960's. There are several 
reasons. The American public is showing increasing 
resistance to mounting taxes. The mushrooming 
demands of social reform programs are producing 
greater competition for the tax .dollars at every 
level of government. And, what is • of great 
potential significance, the taxpaying public and 
their legislators are becoming increasingly 
disenchanted with higher education in general. 
They appear to have been "turned off" by radical 
student behavior, by even higher admission 
standards, by increasingly strident faculty salary 
demands and, most retently, by the shortage of 
jobs for new college graduates^ 

Also in the wind is a growing interest ininnovation 
and experimentation in veterinary medical 
curriculums and teaching methods. Part of this is 
self-generated among progressive faculty members. 
But much of it has been triggered by other 
stimulants such as: 

Austere state budgets which force 
efficiencies. 

Availability of federal funds through 
programs which put a premium on increased 
enrollment.^ • 

The pioneering example of certain 
innovative human medical schools. 

A growing body of literature on medical 
education. 
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The rapid recent development of new 
technology for teaching and evaluation. 

The changing character of the veterinary 
medical profession. 

Student demands for more flexibility, 
objectivity, and relevance in the curriculum. 



What Are the Implications? 

When all factors are considered it seems clear that 
in the 1970's we in veterinary medical education 
will have to take a closer, more objective look at 
ourselves. We shall have to become more 
productive, which is to say more effective and 
more efficient. 

New programs will have to be funded mostly 
through reallocation of resources rather than with 
additional new funds. Program improvement will 
have to be achieved mainly by increasing 
productivity rather than by adding people and 
dollars. 

That is not all bad. Nor is it entirely illogical. In 
fact, it seems to me extraordinarily naive to expect 
that those paying the bill (taxpayers) will happily 
support universities in the style to which they have 
become accustomed simply because we think they 
should. Economists agree that it is difficult, if not 
impossible, to separate price from value. And it is 
the one who picks up the tab (taxpayer) — not the 
vendor (professor) — who must feel that the 
product is worth the cost. 

Of course it would be 'unfair and inaccurate to 
cfaim^that we are being victimized by 
circumstances beyond -our control. Much of our 
current dilemma is of our ovyn making. 

The federal government has responoed with funds 
for increased enrollrrient because we have insisted 
for years (apparently very convincingly) that there 
is a great and growing shortage of veterinarians. 

We appear to be losing the support of some rural 
taxpayers because we are not producing enough 
veterinarians who are willing to enter farm practice 



and who can function effectively in today's highly 
businesslike livestock industry. 

Urbci clients (who also are - taxpayers) are 
complaining about the lack of emergency service 
on evenings and weekends. 

Veterinary students are becoming increasingly 
critical of their education partly, if not largely, 
because they feel that curriculum improvement 
has lagged behind public need. 

Legislators are becoming disenchanted with 
medical education, including veterinary medical 
education, because of what they interpret as its 
excessively high cost and low accountability. 
Whether their view is accurate or not, the facts are 
that they believe it and we have done little to 
change their belief. 

The colleges also are being accused of graduating 
veterinarians who are increasingly mercenary. Both 
clients and older practitioners complain that new 
graduates lack the commitment to service that 
their predecessors had; that they are more 
interested in salary, hours; and fringe benefits than 
in taking care of patients. This of course is an old 
allegation. Whether it is any more valid now than 
in earlier years is open to question. But if it is true, 
there is great danger in its implications for the 
future of veterinary -medical practice. Perhaps it 
would be revealing to have a look at what has 
happened in human medicine where this 
development appears to be further advanced. 

During the past two or three decades there has 
been a striking change in the perception of 
physicians by their patients and, as a result, by the 
public at large. The image of the physician has 
changed from that of friend and counselor to that 
of scientific businessman. This change has been 
promoted by the growing impersonality of the 
doctor-patient relationship. The family doctor 
seems to be disappearing. Very few physicians will 
make house calls; even an illness developing 
suddenly at home now is referred to the hospital 
emergency room where it is treated by a harried, 
youthful, total stranger. 

The remarkable growth of specialization also has 
contributed powerfully to the change of public 
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attitude toward physicians. The specialist of 
course is more competent. But because the 
specialist's patient usually is referred, hence a 
temporary patient, and because of the specialist's 
interest in a narrow field of medicine, he is likely 
to seem less genuinely concerned for the patient as 
a whole person. 

One serious consequence of the growing 
impersonality of human medical practice is the 
enormous increase both in the number of 
malpractice suits and in the dollar values of 
damages awarded. Malpractice insurance rates for 
physicians are skyrocketing. It seems that a 
disgruntled patient is more likely to sue a scientific 
businessman than a friend and counselor. 

The lessons for veterinary medicine to be drawn 
from these developments in human medicine 
should be fairly obvious. The problem may be one 
to be grappled with . primarily by organized 
veterinary medicine. But the role. and influence of 
veterinary medical educators in shaping the 
attitudes of, and clarifying the moral issjes for, 
future practitioners certainly cannot be 
overemphasized. 

Much of what we are doing in veterinary medicine 
was borrowed from human medicine. Part of it has 
suffered in the translation and some of it was 
inappropriate to veterinry medicine in the first 
place. But, on the whole, the example of human 
medicine has served us well in our rapid transition 
from a rustic art to a modern medical science. 

Given the sophistication of veterinary medicine 
today, there certainly is no need for us to ape 
human medicine. But it seems wise 'for us to 
recognize the obvious similarities of the two 
professions and to try to profit from the 
knowledge that/ historically, many trends in 
human medicine have preceded similar trends in 
veterinary medicine. 



r:\ ^.What Should We Do? 

Even if I knew exactly what to do, it would be 
inappropriate to try to be too specific in a 
presentation such as this. But here, for whatever it 
may be worth, is a partial list of trends — some 
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actual, some only wishful — which we probably 
should be supporting or promoting in veterinary 
medical education. Most of these trends represent 
not the abandonment of one position in favor of 
another but, rather, simply the shifting of 
emphasis. 

From the "castle-with-moat" philosophy on 
campus toward a more open, outward look to 
other parts of the university and to off-campus 
constituencies. 

From a scientific . inferiority complex toward 
confidence and par feelings about other medical 
sciences and scientists. 

From emphasis on reversing disease toward more 
attention to preserving health. 

From traditional departmental alignments and 
course sequences toward more functional 
combinations and sequences. 

From spasmodic, revolutionary curriculum 
revision toward continuous evaluation and 
evolutionary change. 

From simply adding to the curriculum (by 
lengthening or packing} toward shortening (by 
culling and condensing). 

From emphasis on teaching toward emphasis on 
learning. 

From heavy dependence on the lecture toward 
more programming. 

From the empirical approach to education toward 
research-based teaching techniques, curriculum 
developm*ent,*and evaluation methods. 

From the "generalist" approach to clinical training 
toward options for accommodating differences in 
student aptitudes, interests, and career objectives. 

Fromvtraditional fact-testing of students toward 
evaluation of student performance. 

From forcing the student into rote memorization 
toward development of student abilities in 
deduction and problem,solving. 
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From small enrollments toward larger enrollments. 
<AII trends are not either desirable or undesirable. 
Some are merely inevitable, hence should be dealt 
with constructively.) 

From the chauvinistic, "what's-best-for-the 
profession" philosophy toward a greater concern 
for the wishes and needs of the consumer (client). 

From a preoccupation with techniques toward 
more awareness of principles. 

From perceiving the D.V.M. curriculum as the 
only preparation required for a career in veterinary 
medicine toward acceptance of the necessity for 
continuous, lifelong education. 

The transition of veterinary clinical education has 
been particularly interesting. Throughout the first 
half ot his century the standards of veterinary 
medical and surgical practice were set by private 
practitioners. In the 1950's the veterinary college 
clinics began a very rapid growth in competence 
which has placed them in positions of leadership. 
The reasons are obvious: increased dollar support, 
improved facilities, the development of 
specialization, and the growth of clinical research. 

In all this we should not discount the influence of 
increased student enrollment, which has justified 
larger budgets and larger faculties, thereby 
providing greater breadth and depth of 
competence and more time for research and self 
improvement. Wtiile many are inclined to view 
increasing enrollments as exerting an undesirable 
influence on the quality of education, the reverse 
is more likely to be true. 

Unfortunately, while the scientific stature of 
veterinary medicine has been increasing at a 
gratifying rate, improvement in veterinary medical 
education has been disappointing. Most curriculum 
change has been only cosmetic surgery — a 
rearrangement of features to give the appearance 
, of youth. Underneath has remained the same, 
aging educational organism. 

Of course nobody ever is completely happy with 
the product of a curriculum committee — least of 
all the most progressive members of the faculty. 
Committees operate on « the principle of 



com promise . Com m ittee recommendations 
therefore seldom represRnt a significant departure 
from the status quo because the most innovative 
suggestions of individual members are bargained 
away in the process of committee debate. As 
Thomas Carlyle said, ''Every new opinion, at its 
starting, is precisely in a minority of one." Our 
colleges of veterinary medicine would have much 
more interesting currtculums if we could devise 
some means for providing safe conduct of a new 
idea through the hostile wilderness of modern 
university bureaucracy.*" 



As veterinarians we are relatively new at the 
business of education. Most of us have a quite 
superficial knowledge of' the literature of 
educational research and our educational 
experiments tend to be highly empirical. 
Furthermore, those of us who are most interested 
in improving teaching and learning also are likely 
to be the busiest with other tasks. As a result, it 
becomes easier to think it up than to look it up 
and we tend to keep rediscovering the wheel. Still 
we are making progress and our ignorances and 
inefficiencies gradually are being overcome. 

A number of blind spots persist in our view of 
improved veterinary medical education. For 
example, we have not taken advantage of one of 
the richest teaching resources in the whole 
system — the students themselves. It should be 
possible to design the curriculum, schedule classes, 
and deploy personnel in such a way as to make 
maximal use of this virtually untapped teaching 
resource. 

We have learned already that an autotutorial carrel 
designed for two students is more effective than 
two carrels designed for one student each. This is 
because of the reinforcing (teaching) influence 
each student has on the other. To carry this 
furthier, upper level students should be able to help 
teach lower level students, to the enormous 
benefit of all concerned. Faculty time now spent 
i n p resenting introductory, elementary, or 
orientative material would become available for 
student counseling and evaluation. At least two 
other benefits would accrue to students who are 
involved in teaching. The person who is required 
to teach is thereby stimulated to learn more 
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accurately. Moreover, through teaching one 
practices communication. And poor 
communication is perhaps the greatest single 
deficiency of professional individuals and 
organizations. 

There is another long-standing blind spot in our 
vision of better veterinary medical education. I 
refer to our contradictory attitude toward 
agriculture. We tell our students that in large 
animal practice they must understand, support, 
and work closely with livestock producers. But we 
do not often behave that way toward the animal 
science departments on campus. We acknowledge 
the importance of genetics, nutrition, and 
husbandry to livestock health, yet we have only 
minimal interaction with those agriculturists on 
campus who have the greatest expertise in these 
subjects. It is ironic that, with our new found 
interest in collaboration with human medicine, we 
are developing reciprocal agreements of a type we 
never considered with agriculture. In view of the 
changing character of animal agriculture, we need 
the help of animal scientists in the training of our 
students, in the design of our clinical research 
projects, and perhaps even in the continuing 
education of our clinical faculties. 



Conclusion 

When one considers what needs to be done in 
order to bring our educational practices up to our 
aspirations, the task seems overwhelming. To begin 
with, we are not certain about what needs to be 
done. Beyond that, implementing such changes as 
we might agree upon is an awesome task. 

As Machiavelli .pointed, out, . . there is 
nothing more difficult to carry out, nor more 
doubtful of success, . . . than to initiate a new 
order of things, for the reformer has enemies in all 
of those who profit by the old order, and only 
lukewarm defenders in all of those who would 
profit by the new." 

Paradoxically, in these days when change has 
become more fashionable, it has become at the 
same time more difficult to bring about because of 
the enormous proliferation of committees, 
councils, and democratic procedural channels. 



The more conservative among us are likely to 
decry "change for change's sake" and to caution 
against "extremes." But there is much of 
Gertrude Stein's contention that "a difference, lo 
be a differen'^e, must make a difference." And to 
really make a difference, changes must be 
extensive — extensive enough to alarm 
conservatives and to discomfort those who have 
settled amiably into the status quo. 

It seems to me that the dangei of our "going too 
far" in educational reform is remote, indeed. In 
the first place, the university bureaucracy would 
not permit it. A truly radical pi oposal would not 
likely survive the nibblings and gnawings of 
democratically selected committees and councils. 
And if by chance it were to make it through in 
recognizable form, the delay would have been such 
that it no longer would seem radical by the time it 
emerged. 

So my admonition is to promote intelligent 
change, to innovate, to experiment, and to not 
worry about "going too far." If you are not 
convinced that our democratic process will keep 
you from going too far, perhaps you can find 
comfort in the knowledge that no change has to be 
a life sentence. If it doesn't work it can be changed 
again. 

It should reassure you also to realize that 
veterinary students are remarkably durable, 
adaptable, and enterprising.. Having survived 
scholastic screenings at the high school, university, 
and professional college levels, they are entirely 
capable of surviving without permanent scarring an 
occasional educational faux pas. Besides, we in 
education habitually credit ourselves with having 
more influence on students than we really have. In 
the final analysis, the individual student decides 
what he will learn. All that the best of us can do is 
to help him make the correct choices — and we 
are not always right. 

I hope that my remarks will not be considered 
anti-intellectual or anti-academic. I consider them 
to be exactly the opposite. It is the penchant for 
probing new frontiers, for synthesizing new 
knowledge out of old, for testing hypotheses, that 
characterizes the truly intellectual scientist and 
distinguishes him fiom the theoretical philospher. 
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Trying must be part of the intellectual process in So I am optimistic about our chances for bringing 

any applied science such as veterinary medical about significant reform in veterinary medical 

education The theories of the Greek philosopher, education. And I am particularly excited about the 

tmpedocles, although erroneous, were accepted as nntnntiai ;nfi..onpn nf thic ev,m««o:..m nr. tnonh;«« 

truths for 2,000years because they were not pcnential mfluence of this symposium on teaching 

^gg^gj and learning in our colleges of veterinary medicine. 
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VETERINARY MEDICAL EDUCATION AND THE UNIVERSITY 



David W, Mullins, Ph.D. 
President, University of Arkansas, Fayetteville 



You have asked me to address myself to the 
general subject of "Veterinary Medical Education 
and the University." Therefore, I shall share with 
you some of my thoughts on the shifting trends 
and changing directions in higher education and 
their possible implications for veterinary medical 
education. 

First, I shall comment briefly on the history and 
accomplishments, the benefits and the needs of 
veterinary medical education in the southern 
region. ^ , 

The Southern Regional Education Board's 
program for assisting in providing regional services 
in veterinary medical education has grown and 
prospered for more than two decades. As you 
know, this program is based on contracts through 
SREB between states which have schools of 
veterinary medicine and states which do not. 
During the first 21 years of the program — 1949 
through 1970 — nearly 8,000 spaces were 
provided by five existing schools for students from 
other states. ' 

The benefits from the program are obvious. The 
schools themselves have received more than 
$11 million from the contracts. The program has 
been a pacesetter in exemplifying the desirability 
and the fruitfulness of regional cooperation in 
providing highly trained professional personnel. 
The program has provided veterinary medical 
education for students who otherwise would not 
have had such opportunity. The people of the 
South and the nation are the ultimate 
beneficiaries. The program also has assisted schools 
of veterinary medicine in maintaining high 
admission standards by giving them a wider choice 
of applicants than one state could provide. 



While the program has developed rapidly and 
successfully, the needs for veterinary medical 
programs and facilities have grown even more 
rapidly and thus, at this point in time, demand 
urgent attention. 

In recent years, the size of pre-veterinary classes 
has increased significantly. In almost every SREB 
state, there are increasing numbers of* "well 
qualified candidates seeking admission to 
veterinary schools under limited state quotas. 
Every contracting SREB state desires additional 
spaces. Students are seeking enrollment in 
out-of-the-region schools because of the lack of 
spaces in the South. 

It is hazardous to predict the extent of future 
demands for professional and technical personnel 
in ariy field. However, there is no doubt that the 
needs (or all kinds of medical and para-medical 
personnel will continue to increase at a rapid pace. 
The problem of expanding the size, scope and 
quality of our health education and delivery 
systems is one of the crucial challenges of today. It 
arises both from the growing demands and the 
rapidly increasing costs of health education and 
delivery. Veterinary medicine, with its impact on 
human, as well as animal health, is ari integral and 
important part of the health delivery system. 
There also is an emerging and crucial role for 
veterir:ary medicine in ecology, marine and space 
biology, oceanography and other areas. 

Therefore, in view of limited SREB spaces — the 
expanding role of veterinary medical 
education — and the projected needs of the 
future — it is obvious that the SREB program 
should be augmented. To achieve this objective, 
the SREB special committee on veterinary 
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medicine, which I had the privilege of chairing, 
recommended to the board that three of the five 
existing veterinary schools in the South be 
expanded and that two new schools be established, 
including the one now under construction at 
Louisiana State University. These steps, the 
committee feels, would meet current demands. In 
view of federal assistance in financing the 
expansion and construction of these schools, the 
committee believes they should be committed 
fully to the concept of regional service. 

While expansion certainly is one of the crucial 
needs of veterinary nnedical education, it does not 
stand alone. Veterinary education, like other 
professional programs, is subject to the changing 
conditions affecting higher education and society 
in general. 

Those of you who have had the opportunity of 
reading the April 1 issue of your Association 
Journal were doubtless impressed by 
Dr. George C. Poppensiek's article entitled, 'The 
Impact of University Reform on Veterinary" 
Medical Education." He makes the point that the 
academic programs of schools of veterinary 
medicine seem to have been less affected by the 
social and ethical revolutions of the 1960's than 
have those of the non-professional' schools. 
However, he concludes that veterinary students are 
no less impatient with the need for changes in the 
adequacy and relevancy of the instructional 
programs. He refers also to the needs for curricular 
revision to reflect the changing needs, of the 
society the profession serves. 

In my view, all higher education stands at a 
crossroads today. 

During the past quarter of a century, universities 
and colleges have been concerned in larn?- measure 
with expansion to care for the influx of students 
following World War II and the Korean War. They 
also have been involved in a vast expansion of 
research and public service progrfims to cope with 
national priorities resulting from the advent of the 
Space Age. 

Enrollment increases are due largely to the 
population explosion and to the ever growing 
percentage of high school graduates electing to 



attend college. The mushrooming of research and 
public service activities is correlated with the 
geometric progression of science and technology 
associated with the Space Age. 

Universities are still faced with the challenges of 
responding to multiplying demands for expanded 
programs of education, research, and public 
service. However, other external and internal 
pressures for improvement of higher education 
arise today from changes in society. The people of 
oi>r nation, and indeed tho world, aro in the 
process of reassessing values, shifting emphases and 
striking out in new directions. They are concerned 
about the impact of new knowledge and new 
ideaj.. They are seeking solutions to such urgent 
problems as the need for international peace, racial 
u nrest, desecration and pollution of our 
environment. They are concerned about poverty in 
the midst of affluence, and the scientific and 
technological upheaval that has altered the 
substance and quality of our lives. Philosophers 
and social scientists for years have been warning us 
that we are not giving adequate attention to the 
social and economic problems of our times and 
that technological developments have outrun our 
political and social processes. Thus, higher 
education, it would seem, has no choice but to 
give attention to these changing conditions and 
shifting priorities. 

In a commencement address at the University of 
Tennessee Recently, SREB President Win Godwin 
has spoken eloquently to this general question and 
I quote: "Higher education today faces new 
social, political and economic realities which 
demand academic reforms, not just more 
academics; efficiency, not just growth; more help 
in applying knowledge to critical public problems, 
not just more research ... We need more 
flexibility in the teaching system, more points of 
entrance, transfer and exit, and more opportunity 
for work and experimental learning." 

It is generally agreed that higher education must 
become more introspective. During the period of 
explosive expansion, high education has not given 
adequate attention to internal evaluation and 
restructuring. 

Therefore, it is time, in my opinion, for all who 
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have leadership responsibilities in higher education 
to do some soul searching. Higher education must 
be more flexible and adaptable than in the past. 
Society demands it. Our programs must be subject 
to continuous planning and painstaking review and 
evaluation in each major division of the university 
and in each major discipline. The rigidity of the 
past is incompatible with the dynamic demands of 
the present. 

We have changed, and are changing, but not fast 
enough. In too many areas we continue to be 
satisfied with the status quo. 

One of the major challenges we face is a growing 
credibility gap on behalf of the publics we serve. 
Gone are the days when higher education was 
inviolable — its programs unquestioned — and its 
decisions unchallenged. Today, questions are being 
asked, programs are being subjected to scrutiny. 
Our decisions are being niore closely examined. 

Financial needs of universities have escalated; so 
has the public demand for more economical and 
efficient use of available resources. One writer has 
referred to this development as "more for 
less . . . higher education's new priority." 

In speaking to this subject, Allen Pifer, President 
of the Carnegie Corporation, which long has taken 
a constructive interest in higher education, 
declares that, quote, "This nation can no longer 
afford the luxury of an unplanned, wasteful, 
chaotic approach to higher education." In my 
view, this is an overstatement of the situation. 
Nevertheless, it does express a view held by a 
considerable segment of our publics. 

Thus, another challenge which faces higher 
education in the immediate future is how it will be 
structured and financed. How will the various 
states structure and coordinate higher education, 
both public and private? How will it be financed? 
What share will the state provide? What share will 
be provided from federal sources? And what share 
will the student bear? In my view, the trend of 
placing an ever-increasing burden upon the student 
should be reversed. Higher education should be 
provided for qualified students at a price they can 
afford without unduly mortgaging their future. 



Another challenge must be the internal 
restructuring of the curricula and programs of our 
universities. All of our program areas — both 
general and professional — could profit from 
careful re-examination. 

There are numerous and, indeed, revolutionary 
proposals abroad in the land today. For example, 
Alexander M. Mood, Oirector of the Public Policy 
Research Organization at the University of 
California, has advanced a much-discussed and 
rather extreme proposal under the title of 
"Universal Higher Education." Or. Mood believes 
that students in the future will attend college 
full-time for only one year and that additional 
higher education will be a part-time activity 
extending over one's lifetime. He suggests that 
every youth attend college for one year whether or 
not he or she has graduated from high^school. He 
believes that the bulk of education beyond the 
first year should be accomplished at home through 
the use of learning consoles and video cassettes. 

Other concepts are being advanced, together with 
experiments to test their effectiveness. 

The "British Open University" is receiving much 
attention as a possible model for some American 
experiments in providing education at a lower cost 
to the student and to the public. The "British 
Open University" emphasizes a greater outreach of 
university programs off-campus, a change in degree 
structures, and more contiriuing educational 
opportunities. 

'The university without wa!!s" in this country 
represents a similar approach to the British Open 
University concept. This model is being tried at a 
number of colleges and universities. 

Patterns of acceleration of degree programs also 
are being urged on grounds of economy and 
efficiency. Credit by examination is being 
practiced to an increasing extent by colleges and 
universities, thus permitting acceleration and 
recognizing off-campus proficiency. 

A large role in the future may be assumed by 
on-campus and off-campus electronic teaching 
devices. This applies also to exploding library 
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technology through which on-campus and 
off-campus access to knowledge will be the key to 
economy, efficiency and acceleration of the 
learning process. 

The Carnegie Commission under the leadership of 
Clark Kerr has taken a penetrating look at the 
whole spectrum of higher education and has come 
forth with some far-reaching recommendations. 

Among other things, the Carnegie Commission 
proposes "less time, more options." The 
commission suggests the shortening of the B.A. 
and M.D. programs to three years. The commission 
believes that the time spent on the way to the B.A. 
degree can be shortened by one year immediately 
without sacrificing educational quality. The 
commission believes that time spent on the Ph.D. 
and M.D. degrees can be reduced by an additional 
one or two years. A -considerable number of 
undergraduate programs and medical school 
programs are experimenting with these procedures. 

In the matter of more options, the commission 
suggests more opportunities in lieu of formal 
college work, more points of entrance and exit, 
stronger emphasis on continuing education 
programs, and opportunities to alternate 
employment and stuuy, such as the "sandwich" 
programs in Great Britain. 

The commission estimates that if all the reforms 
recommended were implemented, operating 
expenditures for higher education could be 
reduced by 10 to 15 percent a year below levels 
that would otherwise prevail by 1980, or by 
$3 to $5 billion a year. In the opinion of the 
commission, construction costs in the 1970's 
could be reduced by one-third or a total for the 
decade of some $5 billion. 



The Carnegie Commission's recommendations are 
receiving wide and often favorable attention by 
governors, by legislators and the general public. 

What does all of this mean for the higher 
education community and for those charged with 
the direct responsibility for professional 
education, including veterinary medical education? 

I am persuaded that leaders and policy makers in 
the field of higher education should examine the 
Carnegie Commission's recommendations with 
care, as well as other proposals and experiments 
that'are being advanced. Not to do so, would leave 
higher education open to charges of smugness, 
inflexibility, and unwillingness to consider new, 
and v\rtiat many feel are viable options and avenues 
for the future. 

The SREB already has conducted a conference at 
Atlanta on June 11, 1972, on one aspect of 
curricular revisions, dealing specifically with 
approaches to shortening undergraduate degree 
work. We can be assured that other regional and 
national groups will examine various innovative 
and experimental approaches to restructuring and 
reshaping higher education. 

I would suggest that some carefully developed 
technique be created by each professional 
discipline to explore these recommendations, 
proposals and innovations. 

It therefore occurs to me that the sponsoring 
groups of this symposium might appropriately 
take steps to explore approaches applicable to the 
veterinary . medical education field. Perhaps a joint 
commission or committee might be created and 
charged with this responsibility. Such a procedure 
might result in recommendations holding promise 
for sound and accelerated progress in veterinary 
medicai education. 
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GRAND ROUNDS IN MEDICAL EDUCATION WITH MEDICAL 
STUDENTS UP AND DOWN THE CENTURIES WITH A 

CRYSTAL BALL 



Thomas Hale Ham, IM.D. 

Hanna-Payne Professor of Medicine and Director, Division of Research in Medical Education 
Case Western Reserve University, Cleveland, Ohio 

Reproduced with additions by permission of the Josiah Macy, Jr. Foundation 
from "The Changing Medical Curriculum." Report of a Macy 
Conference: T.H.Ham, Grand Rounds with Medical Students: Through the 
Centuries With a Crystal Ball, pp. 183-191, 1972, Josiah Macy, Jr. 
Foundation, 277 Park Avenue, New York, New York 10017. 



It is a privilege to report information on medical 
education to this audience, based on an unsual 
happening that occurred about one year ago. A 
disti.nguished private foundtion, that shall be 
nameless, offered me a small grant if I would test 
out their crystal ball with its two new 
attachments; one of them called the 
retrospectoscope and the other called the 
prognosticator. The foundation wanted lo validate 
these instruments for their future use. Since I 
needed data myself in planning ahead, I accepted 
the mission provided I could make the plans 
myself. This freedom was granted and I decided to 
make grand rounds throughout a little more than a 
century, talking directly to one medical student in 
each of four different periods. 

Personal interviews were carried out that brought 
me first hand information from 1870, 1949, 1970, 
through the retrospectoscope, and, most 
interesting information from 1990, through the 
prognosticator. These were refreshing experiences 
indeed because I found the integration of the data 
allowed me to derive four postulates that appear ^ 
essential for learning as well as a series of 
principles for medical education and educators. It 
is hoped that this information and conclusions 
^ may help as we plan ahead in this period which has 



a few problems that are as yet unsolved. Because 
of limitations of time I shall not go into a 
technical description of the instrumentation which 
is based on the laser beam transmission of 
audiovisual communications, retrograde into the 
past and projection into the future. Also, I can 
report briefly that the data obtained by this 
method appears statistically significant within two 
standard crystal balls. 

1870. First I interviewed a student who was 
about to become a physician in 1870, a century 
ago, in a small town in Ohio. He was just 21 years 
of age, was completing his three years of medicine 
and fully expected to be qualified in the 
summertime'.b'Y"the board of censors, allowing him 
to start his own practice. He. was a fine looking 
young man, had a small mustache, and a pleasing 
way of responding. He was a little surprised when I 
said I was calling from 1970, but he accepted this. 
At the age of 18 he had finished his schooling in 
Ohio and had" interviewed the doctor who was to 
become his preceptor. His preceptor was the only 
physician in his community of about 1 000 people. 
An' agreement was reached and. signed by the 
student and the student's parents stating that he 
would serve as the apprentice to the physician 
during three years but would be allowed to go to a 
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medical school for two periods of 18 weeks each 
in his secondhand third year, if all worked out. It 
did work out well because in the first year he had 
a chance to begin reading his preceptor's books on 
anatomy, botany, and materia medica and 
childbirth. He dissected a sheep his first year. It 
was evident that childbirth, fevers, and accidents 
on the farms were important problems for the 
doctor. The student was able to help considerably 
with the care including the surgery, setting of 
bones, making of pills and concoctions. By the end 
of a year and a half he knew many of the people 
in the community, knew the sick especially, and 
began to feel comfortable as he worked in the 
office and went with his preceptor in his carriage. 

With reluctance concerning the value of the 
program, (his preceptor had never gone to a school 
of medicihe). he was allowed to go to a medical 
school in Ohio in the city of Cleveland where he 
had *18 weeks of lectures lasting five days a week. 
There were six professors and he purchased a 
ticket for fifty dollars from each of the professors 
allowing him to attend. One free clinic on 
Wednesday was attended by patients from all over 
that part of Ohio. A good pathology museum, 
skeletons, and one or two cadavers were learning 
aids. He continued his work with his preceptor all 
summer and returned oncy again to take the same 
set of lectures in the third and last year of his 
training. He found the lectures of some value but 
the classes were quite large, being 75, so there was 
not much opportunity for discussion. He did not 
examine any patients but he learned of new 
sciences that were coming. There was no doubt in 
his mind that work with the physician in his town 
was the exciting aspect of his training. He was 
planning to move to a still smaller community 
when he was qualified so he could know all of the 
families. His physician preceptor was a good 
teacher and a respected person in the community 
although he could do relatively little for many of 
the seriously ill patients who came to his office or 
were attended in the home. There was no hospital 
near enough to use* Fevers carried away ten adults 
and seven children in one winter month in 1870 
and abscesses were a major complication, of 
surgery. 

1946. I then asked to see a student in 1946 and 
found that adjustment on the crystal ball was not 



too difficult. I interviewed a student who had had 
four years in a modern medical school and found 
some remarkable phenomena that were recognized 
by him and his classmates. He said he had no 
contact with, the community whatsoever and saw 
practically no patients during the first two years 
except at considerable distance or with a group of 
other students. He found the departments were 
excellent indeed and were almost research 
institutes unto themselves. There were good 
opportunities to observe investigation which had 
brought great advances. These were thrilling. 
Somehow the faculty wanted to cover everything 
because everything seemed so important. The 
available material was excellent but he was 
overwhelmed by it all. He found that departments 
were such tight compartments that they did not 
know what each other was teaching and they did 
not seem to care. This seemed strange to my 
student who said he was buffeted from one 
examination to the other. He had a hard time to 
decide where to go into practice. Certainly he 
would. choose an internship in a university setting. 
It was the only thing. 

He found that grades and class standing made 
competition tough with his peers to get a top 
internship. This made for discomfort with his 
classmates and' the learning was done by the 
numbers. But he said the clinical work was 
excellent and he felt better as he worked with 
children. He wants to do pediatrics, I wished him 
good luck. 

1970. It was now possible to visit a student who 
was enjoying himself in medical school in the 
southern part of the United States and I asked him 
how come he was enjoying it. He said that the 
school seemed to recognize his needs as a student, 
had given him a chance to learn on his own with 
guidance but with considerable freedom. In most 
of the courses the faculty had joined together to 
plan for the student with more than one discipline 
taking part, so that it seemed logical to relate the 
clinical problem and the basic science aspects from 
multiple approaches on the same subject. This 
seemed obvious and permitted him to study on his 
own from materials prepared by the faculty. The 
faculty had given heavy responsibility to him to 
learn on his own and held him to it. He was doing 
much reading and preparation on his own and was 
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seeing patients at the beginning of the firjfyear 
which turned out to be a staggering onallenge 
because he felt so ignorant. However, he said there 
■was little contact with the community and the 
separation of the clinical and the biologic sciences 
was highly artificial much of the time. 

He was glad to say that the students were not 
graded and there was no class rank so that they 
studied together. He found the examinations were 
tough, demanding, kept him from getting the least 
bit soft, because they were graded on a pass-fail 
basis and given right back to him to see. He said 
that the exam was being used as a qualifying 
exercise and a learning approach but it was still 
hard for him to get over his earlier views , that the_ 
exam was a penalty box. He and\his class[riates,j 
wanted to get* out into the community but there 
was no one there to help them very much. The 
faculty were busy with other things. A few of the 
students did reach out to work in other types of 
clinics where they could get guidance. This excited 
the class a lot but didn't seem to excite the faculty 
very much. He was a bright young man, very 
serious, with a beard — a neat beard. 

1990. I adjusted the crystal ball to the 
interviewing of a student in the year 1990. The 
image coming from the future was not nearly as 
clear as the retrospectoscope had brought from the 
past. There was more fading and static from the 
laser probe. However, I will give you what data I 
have, much of it is verbal rather than visual. The 
actual view of facilities and other details will have 
to await improvement in the instrument itseif. 
They could not be seen. My interview, however, 
was with a young man who had begun his career 
for medicine when he was still in high school. He. 
found it perfectly logicaNto begin working with 
patients about once a week to conduct a certain 
number of observations relating to the patients' 
problems. His data contributed to the doctor and 
the other health professionals in a useful manner 
that was completely understood by the patient. He 
found that the patients' problems excited him to 
explore his work in biologic sciences and he was 
able to continue this contact with the patient, and 
with the health professionals, as he did his 
sciences. 



There were programs available that he could work 
through as a series of units on a particular subject 
or system and he mentioned the heart as an 
example because he had studied this with other 
students using assigned materials which were 
developed just for their group's use. There was 
always one faculty member on call, like a resident 
preceptor, and this brought friendly advice to him 
and all of his colleagues who were studying 
together. They knew from self evaluation units 
when they were ready to qualify in a subject and 
then they did so. He elected to do a project thesis 
on the embryology of the heart. 

He participated in the continuity of observations 
of his patients as he attended college and medical 
school. The patient and his family were the units 
of this endeavor. This he found rewarding. He saw 
patients on a regular schedule with his preceptor, 
had gotten to know a considerable number of 
people and the whole health facility where the 
physician worked. He felt at home there. They 
followed the patient in a group practice and 
employed several diagnostic units and referrals to 
hospital specialists when necessary for advanced 
observations or treatments. He liked the feeling 
that the patient was the primary concern. He 
found his basic science knowledge was becoming 
more meaningful to him each year. 

He would obtain his M.D., he said, after qualifying 
in the several biological and clinical requirements. 
He could do this in several ways depending on his 
career choice. The time required depended on his 
program that he worked out with his counselor. 
He planned to extern after qualifying for his M.D. 
in a group practice unit in a community that, 
provided consultation by direct audio-visual 
communications from the clinic to consultants in 
several areas. ■ He found that there was no 
separation of the community, the physician, or the 
experts in the school. Consultation had become 
universal. He followed all his patients to the 
hospital. This young man was only 24 and would 
be ready for practice in one year. I could not see 
him but he had a fine sense of friendliness and 
quiet humor. It was refreshing to hear his many 
enthusiasms. He said there were no wars going on. 



27 



Summary. I think you can recognize that these 
interviews have brought us crystal clear data from 
the grand rounds with four students from 
1870 to 1990 — truly a privilege over these 
120 years. The results might be summarized as 
follows. The 1870 experience, as shown in 
Figure 1, shows how the physician was thrust into 
the needs of the patient and the community. The 
student was an apprentice to him for three years 
being allowed reluctantly to attend two courses of 
lectures in his second and third years. The student 
certainly knm what was important when he 
worked with the preceptor and he knew how he 
was doing as an apprentice because the preceptor 
told him. 

Unfortunately preceptors could not teach all of 
the basic sciences that were developed in the next 
forty years from 1870 to 1910, so that the change 
over of medical schools brought departmental 
teaching with the,. needs of departments 
transcending many others, as shown in Figure 2. 
The student inherited the departmental research, 
the excellence and complexity but was often lost 
as an individual and as a member of community 
medicine. 

Then, in the period 1945 to 1970 the needs of the 
individual student were discovered without 
destroying the departmental structure of the 
medical schcni. The student suddenly had become 
a unifying force because his problems in learning 
became loud and clear in this abundance, of 
excellence. Faculties cooperated to develop 
interdepartmental instruction and curricula to help 
the student. There was guidance by multiple 
disciplines for the student, as shown in Figures, 
There were beginning innovations in pedagogy, but 
there was still separation of basic science and the 
patient, separation from the community and 
considerable separation from the patient. 

By 1990 it appears, from the interview of the 
student, that the needs of, the student, the needs 
of 'the patient and the needs of the community 
had .all three been recognized without the 
destruction of the excellence of the departments. 
Other needs were identified, as shown in Figure 4. 
This meant that multiple problems were addressed 
and were not made to be mutually exclusive. 



There was a continuum of needs that could be 
addressed as problems. The university became 
universal to man and not to its own campus. 
Actually the student found this perfectly normal. 
Big science could continue as could educational 
innovation and the application of these approaches 
to the needs of the patients in their communities. 

Once these several needs were discovered, it was 
possible to agree with one concept, namely, 'The 
fundamental object of medical education is to 
make good doctors/' Also, it became apparent 
that the student could and would take 
responsibility to learn on his own and that he 
would respond as a colleague in learning medicine 
and in patient care if the environment for his 
education was carefully planned. In fact four 
postulates for learning were discovered by this 
brief experience with the crystal ball and its laser 
attachments. It was the medical students. who 
made these clear. These postulates are now 
presented to you, largely as hypotheses, for you to 
test as you go forward in medicine as teacher and 
physician. These postulates appear essential to 
learning, all pervasive in effect, and greater in 
influence than any one curricular effect. They may 
have universal application in learning. 

Postulate No. 1, Caring About the Student. It is 

believed that caring about the student is a process 
that literally rediscovers the student as an 
individual who has a career that is unique. As soon 
as the individual's career becomes the unit of 
education it is recognized that a plan can be" 
developed for the student to learn on his own 
initiative. This is a privilege. 

Postulate No. 2. Informing the Student What Is 
Important. The student demands from his faculty 
sonrie estimate of what is important in the methods 
of learning, in factual material, problem solving 
and patient care. Without explicit definition of 
objectives and competencies expected of each 
student he may be quite lost, frustrated, and 
upset. 

Postulate No. 3. Guiding the Student in 
Learning. Since -the student is the one to do the 
learning, the faculty can devise increasingly 
sophisticated systems for the student to learn on 
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FIGURE 1. 1870 Preceptoral Teaching 
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FIGURE 2. Departmental Teaching 
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FIGURE 3. Interdepartmental Teaching 
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FIGURE 4. Problem Approach to Needs in Medicine 



ERIC 



NEEDS 

of 
Patient 

Community 

Physician 

Student 



Medical 
Sciences 



Education 
Becomes Problem Oriented 



Multiple Disciplines 
Participating 



1970 1990 
Carnegie Report No. 2 



32 



his own but with faculty-student interaction of 
friendly guidance. He wants to know the degree of 
responsibility that is delegated to him in learning 
and in patient care and wants to know how to 
proceed with effectiveness and authority. 

Postulate No. 4. Helping the Student Find Out 
How He is Doing. The student will reach out with 
initiative if he can learn how he is doing at 
frsquent intervals. This can become a key part of 
the day-to-day learning experience in all fields. It 
is the essential feedback system that is needed by 
the learner to know of his own progress. With such 
knowledge he will proceed on his own, assign 
himself tasks, and he will be ready for qualifying 
exams, such as the National Board, that he can 
take in stride. 1 (Figures 5, 6, and 7 extend these 
postulates.) 

We are almost done. We have avoided detail by 
stressing these powerful environmental postulates 
that may set man free as a learner. But before we 
leave the student of medicine, please look through 
the eyes of our future patients because they, too 
will respond to the same postulates, as so well 
described by Frances W. Peabody in 1927.2 

1 . Caring about the patient as a person. 

2. Making clear what is important to the 
patient. 

3. Guiding the patient in his 
progress — friendly guidance. 

4. Telling the patient how he is doing. 

Now, lest we become lost in a humorless society 
and devoured by the overwhelming problems of 
today, tomorrow, and the year 2000 — please let 
me join you in some philosophy that is codified as 
follows for each reference. This section is entitled: 



Law of Pedagogy and Innovation 

The. Reverse Peter Principle or 

How to Be a Failure and Like It «^ 

When two or more are gathered together some one 
will lecture. 



Hard seats lead to intellectual decubitus ulcers. 

Lectures can produce fatty changes of the 
cerebrum, known as cerebral pafe de foigras. 

The student cannot learn it all, cannot discover it 
all, cannot know all the right answers, but he 
surely tries. 

To test the unimportant is too easy, the important 
may defy measurement. 

The swing of the pendulum is better for keeping 
time than'for making policy. 

The problem approach makes us interdepartmental 
colleagues and somewhat honest. Let us try it. 

The needs of the patient and the needs of the 
student can guide us in continuing change. 



Laws for Innovators 

Do not try to iiarvest your crop before you plant 
the seed. 

Have your seed ready for your planting; a seed 
catalogue^ will help. 

Do not plant the seed until the ground is ready. 

Examining the roots every day delays growth of 
the plant. 

Dne person can plant the seed but many are 
needed for the harvest. 

Some plantings take several years to bare fruit. Do 
not hurry them. 

The Peter Principle^ and How to Reverse It 

The Peter Principle is needed to guarantee 
bureaucratic degeneration and academic stenosis. 
To preserve this incompetence and confusion, all 
actions must proceed rapidly without listening to 
any one. This produces high antibody titers in all 
participants. Conversely, working from the bottom 
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FIGURE 5. Problem Approach to: Needs of the Teacher 
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FIGURE 7. Communications, Learning, and Evaluation 
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up, you can hear better. This is known as the 
Reverse Peter Principle or how to be a failure and 
like it. Ffom the bottom up (abbreviated FBU) 
can be measured in units of listening before acting 
(abbreviated LBA). One LBA unit will neutralize 
one Peter Principle unit with development of a 
cloudy precipitate and some heat of reaction. The 
solution is quickly cleared by an excess number of 



L8A units. This Reverse Peter Principle (RPP) is a 
delightful state. Optional courses can be arranged. 

Use of the crystal ball and its laser attachments 
and support for these grand rounds are accepted 
with appreciation from the- granting agency. (All 
rights to the above laws and prinliples are 
copyrighted but you have the right to copy.) 



FOOTNOTES 



1 IVIainstreams of Medicine. Essays on the Social and Intellectual Context of Medical Practice. 
Edited by Lester S. King, M.D., Ham, T. H. "A Clinical Investigator Looks at Medical Education. The 
Discovery of the Medical Student as a Responsible Colleague." pp. 99-100. University of Texas Press, Austin 
and London, 1971. 

2 Frances W. Peabody, The Care of the Patient, J.A.M.A. 88:877, 1927. 

3 Burpee's Seed Catalogue of Pedagogic Varieties lists over 425 species. Many hardy perennials are 
available for distribution by the author (T. H. Ham). Listings available on request. 

4 Man is promoted until he finally, reaches the level of his incompetence. Peter, Laurence J. and 
Hull, Raymond, The Peter Principle, William Morrow and Company, Inc., New York, 1969. 
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MINICOURSES — WHAT ARE THEY? 



S. N. Postlethwait, Professor 
Department of Biological Sciences, Purdue University, Lafayette, Indiana 



Dr. Postlethwait's presentation was drawn from the following manuscript* and the study guide entitled 
"Minicourses — What Are They?" by S. IM. Postlethwait and Frank Mercer copyright 1972., Purdue Research 
Foundation, Purdue University, Lafayette, Indiana. Copies may be obtained by writing the author. 

STUDENTS ARE A LOT LIKE PEOPLE! 



The technological revolution is met with mixecf' 
-emotions by both educators and students. The 
reason is relatively simple — people do not like to 
be replaced by machines and people do not like 
machines as a replacement for people. Education is 
more than an information dispensing and 
absorbing process. It requires a comradeship of 
sharing and exchanging of experiences and an 
excitement that grows from common interests and 
hopes between teacher and student. Most of us can 
trace our interest in a specific topic to the 
inspiration derived either directly or vicariously 
from some other human being. 

Does this mean that one must reject the utilization 
of all technical devices in the development of an 
educational system? Far from it! Properly used 
equipment may very well enhance the personal 
relationship between student and teacher. Some of 
our most effective and powerful lessons are 
learned from teachers whom we have never seen 
but with whom we have a fellowship derived 
through some spanning medium such as the 
written word. Clearly this written 'bridge' between 
teacher and student has been a great boon to all of 



mankind and, until recent years, has been our only 
link to many of the great teacherfJ cf the past. 
F 0 rtunately, technology has expanded our 
potential for even more intimate access to great 
teachers and with greater facility. The dimensions 
of audio and visual now can be preserved and 
retrieved and fantastically convenient vehicles. The 
imagination of man for utilization of these devices 
individually or in combination with the written 
word and/or tangible items is one of the major 
limiting factors in education today. 

If one uses the education model proposed by 
Hopkins of a 'student on one end of a log and a 
teacher on the other' the role of technological 
devices becomes more clear, and less foreboding to 
many of us. The purpose of technology in this 
context is to 'capture' to the greatest degree 
possible the events or activity between the 'good 
teacher' and student in a one to one relationship, 
so that the product can be duplicated to 
accommr"*ate many students in • a 'close 
approximc.on of the original situation. With 
today's audio and visual devices it is possible' to 
involve the student with nearly every exposure to 



* Postlethwait, S. N. "Students Are A Lot Like People," Journal of the British Universities Film 
Council, No. 8, January, 1972. 



the subject conceivable. The program can contain 
tangible, printed, audio and visual materials in any 
combination in which the 'good' teacher wishes to 
use them. The only limitation is the capacity, of 
the teacher and student to relate to the simulated 
situation. 

In 1961 at Purdue University a program was begun 
which has been called the Audio-Tutorial system. 
The basic philosophy of this system is very simple, 
A 'good' teacher is asked to assemble the items he 
would use to teach one student and, while sitting 
among these items, to record on audio tape the 
conversation he would have with one student as he 
tutored that student through a sequence of 
learning activities. The product — the tape, 
tangible items, visuals and printed materials can be 
duplicated as many times as necessary to 
accommodate any number of students. Obviously 
the program produced in this way will be limited 
by the cleverness of the teacher but the corollary 
is also true — a clever instructor can intimately 
involve the student in important and useful 
learning activities. The student now has access to 
the clever instructor in more ways than through 
the written word. Subtle communication through 
connotations by inflections in the voice are 
■ provided by the audio tape and the tangible, visual 
and printed materials, assembled can exhibit the 
ifull skill of a great teacher to involve a student in a 
sequence of learning activities or a 'symphony of 
learning.' 

The Purdue program evolved slowly. The first 
programs were mere lectures on tape and relied on 
audio as the sole medium of communication. 
Later, other media were added including tangible 
items (live plants, models and equipment for 
experiments), printed materials (textbooks, study 
guides and journal articles) and visuals (2x2 slides 
8 mm film and photographs). Study programs 
were set up in booths in a learning center which 
was open from 7:30 a.m. until 10:30 p.m. Monday 
through Friday. Students came in at their 
convenience and spent as much time as necessary 
for them to master the lesson. An instructor was 
available at all times to help students on an 
individual basis if necessary. This study session was 
referred to as an independent study session (ISS); 



Two other study sessions were included in the 
'system: (1) a general assembly session (GAS) and 
(2) a small assembly session (SAS). 

The GAS was scheduled on a weekly ba^isfor one 
hour. It included several hundred students in a 
large lecture hall and involved them in the kinds of 
activities best done in a large group. Specifically, 
this assembly was used for major tests, long films, 
help sessions and an occasional lecture. 

The SAS included eight students and an instructor 
and was scheduled on a weekly basis and used 
primarily for short written and oral quizzes. It 
served in an administrative role and for the 
identification of students with a specific instructor 
in the large unstructured course. 

The course structured in the above pattern 
provided a full range of learning activities and 
situations. The ISS permitted students to enjoy 
, some important features of a learning system that 
are not commonly available in a conventionally 
taught course. Some of these include: 

1. Repetition — There is tittle question but 
that the nature of many objectives required 
repetition for their achievement. However, 
repetition ought to be. engaged in in an 
intelligent fashion and adapated to the 
individual needs of a particular student. In a 
course with 500 students the teacher cannot 
possibly make the adjustments in repetition 
for individual student need. Only the 
student can determine intelligently how 
much repetition is necessary. 

2. Concentration — Most classrooms are not 
organized to permit students to conceritrate 
during their study. Students are distracted - 
by one another, and other dissociated events 
which may be occurring tend to distract the 
student's attention from the subject at hand. 
The audio-tutorial system permits the 
student to isolate himself from the 
surrounding environment by covering his 
ears with the earphones and by the use of 
othei^edia to reduce his awareness of his 
surroundings. 
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3. Association — :ln a study of plant science 
the major objective is to learn about plants. 
It makes sense therefore that a study of 
plants should be conducted where plants are 

- available for observation. Diagrams, charts, 
, models, photographs, and other such devices 
should be a 'means to the end' so that 
students' attention is directed to the actual 
plant. The audio-tutorial system provides an 
opportunity for the student to have an 
object available at the time he reads about it, 
does experiments, etc. 

4. Appropriate sized units of subject 
matter — People vary considerably in the 
amount of subject matter that can be 
grasped in a given amount of time. 
Programmers have demonstrated that most 
people can learn almost anything if it is 
broken into small enough units and the 
student can take time to become informed 
about each unit before proceeding to the 
next. Any program of study therefore 
should provide each student with an 
opportunity to adjust the size of tha unit to 
his own ability to assimilate the information, 
so that those who can absorb large quantities 
of information may do so in an unrestricted 
fashion, whereas others who must proceed 
more slowly are permitted to do so. The 
audio-tutorial system allows the student to 
proceed at his own pace and to break the 
subject matter into units commensurate with 
his ability. 

5. Adapt the nature of the communication 
vehicle to the nature of the objective—It is 
logical that no simple vehicles such as 

~ lecturing or a text book can achieve the full 
spectrum of objectives for a complex 
subject. The student's experiences should 
not be confined to any particular vehicle as 
film, audio tape, text book, or a lecture. In 
cases where the development of a procedural 
skill is necessary, there is no substitute for 
the student doing this procedure himself. A 
properly structured course, therefore, would 
carefully define objectives and not try to 
mold objectives to fit a favorite medium 
(lecture, for example) but instead would use 
the medium best adapted to the nature of 
the objective. 



6. The use of multi-media-— Individuals differ 
in their responsiveness to different kinds of 
communication devices. Some people learn 
well through reading, some nan learn best by 
auditory communication, and others can 
learn best by literally handling specimens 
and performing experiments. The 
audio-tutorial system thus provides an 
opportunity for subject matter to be covered 
in a great variety of ways with the student 
exploiting the medium which communicates 
most directly and effectively for him. 

7. Finally, and most important, the integration 
of learning activities and situations — It 
stands to reason that if learning events are to 
be complementary and to have some 
relationship, they should be brought into 
close proximity and properly sequenced. 
The conventional structuring of a lecture, 
recitation, and laboratory does not take this 
into consideration but rather may expose a 
student on Monday to a lecture concerning a 

, given subject; perhaps on Wednesday the 
student does experiments related to that 
subject; on Friday a recitation will involve 
the student in some exposure to the subject; 
and then on Sunday night, late, the student 
m.ay read on this subject from text. The 
audio-tutorial system permits the student to 
bring all of these learning experiences into 
an integrated sequence so that each learning 
event may enhance or complement the 
adjacent ones and thus result in a 
'synergistic' effect. " 



The individual nature of human beings cannot be 
over emphasized. Any good educational system 
must be based, on the fact that 'learning must be 
done by the learner.' It must be involving the 
student in the process and must always pro\/^ide a 
high degree 'of flexibility and adaptation^ to 
individual needs. However, superimposed over this, 
quest for iridivtduality is the dependence of each 
student on teachers to guide, facilitate and 
stimulate him to engage in appropriate learning 
activities. Today's technology provides new 
dimensions to accomplish this and, when these 
new tools are used properly, they provide more 
intim.ate access to the teacher—they do not 
dehumanize! 
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MINICOURSES IN VETERINARY MEDICAL EDUCATION 

R, Brown, D.V.M, 

Chairman, Department of Veterinary Medicine and Surgery, College of Veterinary Medicine 

University of Missouri, Columbia 



Dr. Postlethwait has described and defined the 
minicourse concept. My task is to call attention to 
present efforts of veterinary educators to place 
this type of educational methodology "on the 
line" and persuade you of the need for its wider 
adoption by veterinary educators. I could have 
circulated a letter to each school asking them to 
detail existing programs. It seemed more logical to 
conserve time and postage by using those items 
presented at Detroit last year and those to be on 
display at New Orleans this July. Politically it gets 
me "off the hook" in the decision making process 
of what to include and what to omit in the brief 
time allocated for this presentation. The titles will 
not be included in the proceedings as they are 
already published in an easily found source, that 
being the 1971 and 1972 convention issue of the 
Journal of the A.V.M.A. 

Because of the numbers of the titles shown, my 
format of presentation will be with two projectors 
and a sound tape. The madness in my 
methodology will be apparent within a few 
minutes. . ' 



Presentation of Slides (76) 
and Audio Tape (6 minutes) 

This presentation should have, in the terminology 
of the professional educator, caused a "change of 
behavior" in you, "the students," attending the 
5thL Symposium on Veterinary Education. You 
should be confused, apprehensive, awed and 
collectively of one mind asking the question 
"What-in-hell is he wanting us to know?" If this is 
indeed your mood, then you arer therefore, 
kindred spirits to a faculty member trying to 



organize a course with 192 contact hours allocated 
and 900 hours of information to transmit. His 
tools are textbooks written for his peers and hot • 
the student, classrooms designed as boxes and not 
learning centers, audio-tutorials developed by 
others that he cannot afford and inadequate time 
to devote to preparation. Lack of agreement 
among his peers (national and local) as to tha 
capabilities of the end product and the over 
present specter of National Board Exams adiJ to 
his burden. 

You are also kindredjpirits to the student. If you, 
who have responsibility for a segment of his 
education, are confused these days;^imagine his 
dilemma. You ask him to review, skim, survey, 
smell, practice, see, hear, write, analyze, present, 
attend, care for, summarize and place in a 
meaningful perspective, hundreds of "bits" of 
knowledge and skills. 

You may think l am discouraged, dismayed, bitter 
and hypercritical of present educational 
methodology in veterinary schools. I may be awed 
but not discouraged by the multiple methods, . 
confused but not dismayed by present lack of 
program definition, bone tired but not bitter ^ 
about our progress, and constructive but not 
hypercritical of our collective efforts here to 
improve the educational process. Never before has 
this number of veterinary edujcators gathered 
together for the express purpose of methodology. 
It warms the heart of one who for twp decades has 
pleaded "the school is for the children." 

What then are the solutions to the problems we all 
face? From my view as a former student, teacher, 
practitioner and administrator an order of 
priorities emerge: 
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1 . Identify, encourage and reward those unique 
individuals who can survey, sort, reorganize, 
pr(?sent, evaluate and measure their subject 
matter area and reduce the amount of 
available material to a quantity realistic to 
the credits and hours allocated. 

2. Expand the efforts of species, systems and 
discipline-oriented groups in an effort to 
further identify what is essential for a 
veterinary student to master and delete that 
portion more suited to post-graduate and 
peer group needs. 

3. Establish the format for exchange of 
educational materials including test 
questions.. The expanded base of . the 
AAVMC, if approved here during this 
symposium and adopted in New Orleans in 
July, will provide the mechanism by 
identifying one individual at each school 
who could serve as a coordinator. Inherent 



in the expanded organization, is a Journal of 
Veterinary Medical Education. 

4. Write a series of textbooks tailored to the 
needs of the student rather than the authors', 
peer group. The perfect example is the new 
"Guide to the Dissection of the Dog" by 
Evans and de LaHunta. Those authors are to 
be congratulated for a job well done. 

5. Use the learning experience of this 
symposium to initiate a minicourse approach 
to veterinary education. This will cause us to 
more clearly state our objectives, select 
media on an effectiveness-cost ratio, and 
provide for the inherent variation in student 
learning ability. 

This brief overview of the profession's efforts to 
utilize new technology and methodology is now 
concluded. May the 5th Symposium be a roaring 
success! 
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MINICOURSE PLANNING 



A. R. Allen, 1\1.S. 

Director, Medical Instruction and Communication 
School of Veterinary Science and Medicine 
Purdue University, Lafayette, Indiana 



Would you start to produce a movie without a 
director, without actors, without a technical crew, 
reliable equipment, potential audience, estimate of 
probable success, or, most of all, without a script? 
If you were making home movies your answer 
would probably be yes.' If you were producing 
feature movies for commercial release, (unless you 
were Andy Worhol) your answer would be no. The 
parallel between the differences in your answer 
and the differences in the quality of the two 
movies (three if you count Worhol's flicks) is 
obvious. The same line of thought can be applied 
to wciting (books vs. diaries), music (operatic . 
recording vs. singing in the bathtub), speech 
(casual conversation vs. oration) or cooidpg, 
paintings, and even mm/courses. Gerrerally 
speaking the quality, the effectiveness is directly 
proportional to the planning and care in 
production. 

The traditional lecture and demonstration will also 
improve proportionally to planning and care in . 
production but, unfortunately (or fortunately, as 
the" case may be) they are not generally recorded 
and are not subject to repeated reuse as are 
movies, books, and records. If a movfe is bad the 
first time, you see it--it only gets worse with 
repeated viewing. Minicourses, if for no other 
reason, deserve speciaLptanning and care in 
reproduction since whet we say will be there to be 
heard over and * over and over ... We find 
ourselves in the publication business, whether it be 
local, limited or universal. 

Minicourses are expensive! They are expensive in 
terms of time, equipment, space, materials and 
personne!. They can solve some problems oL^l 



scheduling, individual differences between 
students, student-faculty contact time, etc. and 
yet may create other problems of logistics, filing, 
. retrieval and scheduling perhaps just as great. 
Therefore, planning a minicourse should also 
include consideration of the advisability of 
selecting an alternative approach. If there is 
validity in the recommendation that a good 
learning situation is "a student on one end of a log 
and a good teacher on the other end of the same 
log" (and there are many cases in veterinary 
medicine where we approach this situation) then 
perhaps everythi .2 need not be a minicourse. 
There are too inany things that need to be 
programmed to waste time and effort on those 
that do not. 

In their Study Guide, Minicourses — ^What Are 
They? (Purdue Research Foundation, 1972), 
Postlethwait and Mercer outline a development 
sequence for Minicourse production. Their 
approach is both logical and sound and is 
presented here in outline only. The author's 
detailed explanation is given in their publication. 

It is interesting to note the choice verbs that 
appear in ihe outline. They are: develop, evaluate, 
state, construct, select, storyboard (used as a ■ 
verb), assemble, rehearse, specify, [Drepare, 
transcribe, prepare, evaluate, transcribe, revise, and 
finalize. Suppose that much effort was placed on 
the preparation of any presentation, he it 
minicourse or other. 

From the production standpoint the storyboard 
(step six in the Initial Development Stage) is very 
useful. The storyboard is a technique used to 
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Modified from: Postlethwait and Mercer, Minicourses - What: Are They ? 
Purdue Research Foundation, 197 2 



Overview 





PHASE 


I 


II 


III 


IV 


. \ 

Operation y 


Preliminary 
Deve l^pme nt 


Pilot 
Testing 


Field 
Testing 


Publication 


Product 


Preliminary 
Vers ion 


Trial 
Version 


Field 
Version 


. Final 
Version 



(Most all of our programs in the Veterinary School at Purdue fall 
in Phase T or II. Many will never go beyond this stage because 
they are constantly being modified and probably will never be 
considered for outside use.) 
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sequence instructional materials in the planning 
stage. It provides a means for shifting cards 
depicting the various elements of a program until 
the most suitable sequence is achieved. It provides 
flexibility for adding or deleting items easily. It 
also provides a very suitable device for multiple 
input from two or more people working together. 
It can provide a visual concept of the temporal 
relationship between the various elements of c 
program and it can serve as a production guide 
throughout the program development. And it is 
inexpensive! This is the step where major 
modifications should be made. Changes in 
production are very costly. 

In the Veterinary School at Purdue we like to 
develop a production script from the final 
storyboard arrangement. This script can be 
duplicated and given to all production staff. It 
facilitates coordination of activities and eliminates 
duplication of e'ffort. 

Planning should also include a realistic production 
time table with a general meeting of all technical 
staff outlining the overview of total program. In 
depth discussion and understanding about 
production capabilities and limitationsi is very 
important. 

One should exercise care about copyright 
infringement. This is too often discounted by 
individuals in their enthusiasm for producing a 
program. The "fair use" interpretation of the 
Copyright law for "scholarly purposes" may not 
hold when it comes to multiple copies rather than 
single classroom projectiles. This consideration 
becomes even more important when we receive 
remuneration for our efforts in developing a 
program which contains copyrighted materials 
without releases. Recorded music is now covered 
by the copyright law. Ideally, planning should 
allow time and money to provide original, 
accurate, custom made materials, produced by 
professional medical illustrators and 
photographers. 



When motion pictures are needed, production 
schedules should be planned to provide time and 
money for quality production. Portable video tape 
recorders often prove useful in working out 
camera angles and action without the expense and 
time loss in shooting test footage. The instant 
replay capabilities provide an excellent means of 
motion picture rehearsal. 

Try to plan an easy and natural approach to your 
audio tape but try to avoid folksiness and corn. 
iJse only those jokes and expressions that can bear 
being heard over and over and over , . . ^ 

Plan a flexible format that will iend itself to 
modifications and revisions. Synchronized 
slide-tapes, filmstrips, and slide-records should be 
avoided, at least in Phase I. 

Consider the logistical limitations of the facilities 
you have for presenting the materials. One lab for 
one course offers no problem but scheduling 
minicourses into multidisciplinary laboratories 
generally means competition for time and spaceV 
Pian the course in short segments 
(10 to 15 minutes) so students can move through 
segments quickly. A fifty minute program divided 
into five segments will allow five students to be 
working sfmultaneously. Where as the same course, 
unbroken, only allows one student to work on it 
at a time. 

Even though Postlethwait and Mercer do not use 
the term publication until Phase IV in their 
Mincourse Development Sequence, it could be 
interpreted as beginning with Phase I . It is a matter 
cf degree or extent of publication. We may be 
only publishing on the local level but when we 
start to record and formalize our concepts into 
minicourses, AT programs, movies, slide-tape 
lectures or in any other format that is for others to 
utilize, we are publishing. There have long been 
established rules governing publication — both 
legal and ethical.. The fact that we are now 
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becoming the publisher as well as the author does 
not relieve us of our responsibilities to these rules. 
In fact, it increases the responsibility. 

In our excitement to develop minicourses and 
other programmed instructional materials there is 
too often the temptation to lay aside these rules 
and also considerations of quality control in the 
name of expedience for scholarly purposes. This 
temptation and practice could be avoided if the 



initial planning included considerations for proper 
financing and realistic estimates of production 
costs and time. 

IJ we include these considerations in our initial 
planning, then perhaps we will be closer to 
producing minicourses that can be used over and 
over and over . . . Even though we have a 
captive audience we owe them more than home 
movies and homemade productions. 



THE KEY PERFORMANCE OBJECTIVES 



R. E. Mager, Ph.D. 
Mager Associates, Inc., Los Altos Hills, California 



Doctor Mager's presentation was drawn^fronn his books entitled: Preparing Instructional Objectives, 
copyright 1962, and Goal Analysis, copyright 1972, Fearon Publishers/Lear Siegler, Inc.. Education Division. 
6 Davis Drive, Belmont, California 94002. 

These books were provided each pre-registrant as pre-conference reading materials. 

Following Doctor Mager/s formal presentation the conferees participated in small work group sessions 
where mini-course topics were chosen and objectives written. The participants then re-assembled in general 
session for a question and answer period with Doctor Mager. 
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MEDIA — WHICH ONE 



WiUiam L. MiUard, Ed.D. 

Director, Office of Instructional Media Services, School of Medicine 
University of Southern California, Los Angeles 



There have been times when each of us has felt 
dissatisfied with our attempts to select the 
appropriate media for an instructional 
presentation. Which medium is best for which kind 
of teaching or learning activity? Can we develop a 
classification system or "cookbook" approach to 
- the listing of an appropriate medium or 
combination of media which is best suited for a 
given subject, instructional procedure, or learning 
experience? 



Research Findings 

Over the past half century a nurr/jer of studies 
have been conducted in an attempt to establish the 
relationship of various media to specific types of 
learning outcomes generally classified under the 
heading of cognitive, psychomotor and affective 
learning. The results have been somewhat 
disheartening in terms of generating media 
selection criteria which can be easily applied to 
operational instructional development problems. 

Although media have consistently demonstrated 
their ability to communicate effectively and 
efficiently in a variety of. teaching/learning 
situations, there has been no clear-cut superiority 
of one particular medium over another in 
producing satisfactory learning outcomes. This 
generalization does not, however, discredit the 
findings of specific studies which do show that 
under certain specific circumstances a particular 
medium may perform well in one instructional 
situation but be unsatisfactory for a subsequent 
instructional event within the same instructional 
program. ♦ 



In addition to the lack of appropriate guidelines 
for the selection of media, our task is further 
complicated by the wide range of media choices 
available. Few of us have had the kinds of 
operational experiences which enable us to feel 
confident about selecting the media which will 
best i-dcilitate efficient and effective 
communication within the context of a given 
teaching/learning process and related instructional 
environment. 

Where then can we turn for guidance in relating 
media to the instructional and learning tasks to be 
performed? The media research previously cited 
provides a clue. These studies suggest that the 
effectiveness of an instructional presentation or 
program may depend more upon the learning 
message design than the particular medium used to 
communicate' the information to the learner. This 
is to say that desired learning outcomes may be 
achieved equally well by any number of media, 
providing factors such as learning objectives, the 
instructional methods, and learning activities are 
optimized. 



Instructional Planning . 

This leads us to the recognition that there does 
indeed exist a body of scientific knowledge which 
can be directly applied to the development and 
implementation of new and improved educational 
programs. The instruction design model (Figure 1)/ 
developed by Or.Jerrold Kemp, reflects a 
systematic approach to educational programming. 
Each one of the steps should be included in the 
process of creating the final educational product. 
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Instructional Design Model* 



Topics 
General P\irpose 



Student Characteristics 



Learning Objectlvei 



Subject Content 



Pre- test 



Teaching/Learning 
Activities and Resources 





Support 




Service 



Evaluation 



Figure 1 



The Instructional Design Model developed by Dr, Jerold Kemp, 
Coordinator, Audiovisual Production Services, San Jose State 
College, San Jose, California, 



Kemp, Jerold Instructional Design^ Fearon Publishers, 
Belmont, California, 1971 
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The significance of this approach is that it focuses 
on learning rather than on teaching. Clearly stated 
and measurable learning outcomes are defined 
first, followed by the designation of content, the 
specification of teaching/learning activities and 
resources, and the assignment of support services. 
Essential to the success of this instructional design 
approach is the systematic evaluation of each step 
in the developmental process plus the assessment 
of student performance once the instructional 
program becomes operational. 

There are literally dozens of similar instructional 
planning and design models in existence, all 
presenting essentially the same basic steps, perhaps 
with moie or less detail and in a different 
sequence. Since the focus of our discussion is on 
the area of media selection, the diagram in 
Figure 2 presents a simplistic ordering of the steps 
involved in instruction/learning program 
development. 



instruction/Learning Modalities 

The identification and specification of the 
instri :ion/learning modalities are prerequisite to 
media selection. The modalities fall into three 
basic classifications: 

1. Exposition--typified by the 
lecture-demonstration usually involving a 
one-way communication from teacher to 
student, supported by the use of media. 

2. Independent Study — typified by printed 
material involving student interaction with 
the media. 

3. Interaction — typified by small group 
d iscussion i nvolving student-instructor 
interaction supported by media. 

These instruction/learning modalities emphasize 
the role of the teacher as designer of instructional 
presentations, creator of learning experiences and 
as an individual counselor for the student. 



Emphasis is placed on the identification of the 
needs, interests and concerns of the learner in the 
development and implementation of learning 
programs. Stress is placed on the kind of behavior 
exhibited by the learner following instruction and 
the development of suitable means for increasing 
feedback and assessment of learning outcomes. 

In effect, we are talking about the strategies and 
techniques involved in the development and 
implementation of these modalities. Strategies 
represent our plan for reaching the stated 
objectives and are composed of the methods and 
procedures which in our judgement will insure that 
the learner does reach the stated objectives. For 
example, we may choose to provide the student 
with all of the cues needed to accomplish a 
learning task. This strategy could be implemented 
by lecturing, giving a demonstration, or assigning a 
programmed text. On the other hand, we could 
elect to give the student no cues and allow him to 
discover ways of achieving the learning objectives 
by a problem solving technique. Figure 3 presents 
a simplified functional classification of 
instruction/learning modalities. 



Media Resources 

The act of communication takes place when two 
bodies exchange information through the use of 
some medium. In order to improve the act of 
communication, we must seek to improve either 
the bodies which exchange the information 
(learner, teacher, or device) or the medium 
through which they communicate. 

Audiovisual materials and devices have ordinarily 
been employed as aids to the instructor in 
presenting information to learners. The lecture 
method ' of presentation, supplemented with 
projected images and demonstrations, is the most 
common example of this use of the audiovisual aid 
approach. 

Instructional media, on the other hand, are 
resources which present a complete body of 
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INSTRUCTION/LEARNING PROGRAM DEVELOPMENT MODEL 



OBJECTIVE 



INSTRUCTION/LEARNING 
MODALITIES 



CONTENT 



MEDIA RESOURCES 



EDUCATIONAL 

PROGRAMMING 

APPUCATIONS 



EVALUATION 



FIGURE 2 



FUNCTIONAL CLASSIFICATION OF INSTRUCTION/LEARNING MODALITIES 



MODAUTY 
Exposition 



Individualized 
Learning 



Interaction 



TECHNIQUE 

Lecture 
Demonstration 



Programmed instruction 
Problem solving 
Laboratory practice 
Assignment 
Mediated presentation 

Discussion 
Conference 
Seminar 
"Deba te 
Workshop 



APPUCATION 

Introduce new subjects and 

activities 
Motivate students 
Present new developments 
Present specialized resources 

(All listed applications) 



Review 
^Clarify 
Correct 
Reinforce 
Synthesize 



FIGURE 3 
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information, are largely self-supporting, and are 
primary rather than supplementary resources in 
the teaching/learning process. 

Although we recognize that more often than not 
these terms are interchanged in common use, we 
need to keep in mind that media implies the 
combined man/device/material interface with the 
learner and includes the concept of audiovisual 
aid. Our task now is to look at the media options 
available to us, identifying their characteristics and 
subsequently establishing some type of operational 
criteria by which we can select media with the best 
chance of meeting specific instruction/learning 
requirements. 

The seven primary media classification categories 
and some examples of each are: 



1. Realia: Specimens, objects, people, events, 

1 . demonstrations. ' 

2. Printed: Text, workbooks, captions. 

3. Still Visual : Photos, slides, overhead 
transparencies, filmstrips. 

4. Motion Visual: Motion pictures, television. 

5. Audio: Tape and disc recordings. 

6. .Programs: Programmed Instruction, 
Computer-Assisted Instruction, combined 
media. 

7. Simulation: Games, models, 
demonstrations. 



Each of these categories can be further defined and described in terms of their instruction/learning application 
characteristics. This simplified listing is presented to illustrate an approach which could be used in the 
development of useful data required in the media selection process: 



Advantages 



Limitations 



1. Realia 

2. Printed 



Present reality 
Permit use of all senses 

Low cost 

Transportability 

Accessibility 



Availability,. 
Accessibility 

Reading skill dependent 
Use of only visual sense 
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3. Still Visual Ease of sequence change . 

(except filmstrip) 
Transportability 
" Ease and speed of preparation 

4. Motion Visual Close to reality 

Capable of depicting motion 



5. Audio Transportability 

Relatively inexpensive 
Ease of equipment operation 

6. Programs lnstant,feedback 

May be adapated to individual 

7. Simulation Close to reality 

Often inexpensive 



Use of only one sense. 
Projection equipment required 
(except photo) 



Accessibility 
Expense 

Equipment expertise required 
Use of only audial sense 



Expense 

Lack of available materials 
Lack of available materials 



Research suggests that a combination of media 
may be more effective in enabling learners to 
achieve objectives by providing for differences in 
learning styles. In general, learning efficiency and 
effectiveness is increased when media are 
employed in teaching/learning events which 
closely resemble the nature of the final student 
performance requirement. 

Figure 4 is a media decision chart indicating the 
points at which discriminations between media 
may be made within the still visual, motion visual, 
and audio categories. 

Our final step is to identify a set of selection 
factors which we can use in conjunction with the 
data provided in the form of learning objectives, 
content, instruction/learning modalities and media 
characteristics. Gerlach and Ely identify five 
selection factors which should be considered: 

1. Appropriateness — Is the medium suitable 
to jccomplish the defined task? 

2. Level of sophistication—Is the medium on 
the correct level of understanding for my 
students? 



ft, 



3. Cost — Is the cost . worth the potential 
learning from this particular medium? 

4. Availability — Are the material and 
equipment available when I need them? 

5. Technical Quality — Is the quality of the 
material acceptable? Readable? Visible? 
Audible? 



Summary 

Media are resources which can provide the 
conditions of learning for the acquisition of 
knowledge, skills, and attitudes. Media selection is 
one step in a systematic approach to instructional 
planning and design. 

The criteria for media selection are dependent 
; - upon the learning objectives, content, 
instruction/learning modality, and instructional 
setting. While research has established some 
guidelines for determining^what medium is most 
appropriate for particular learning situations, there 
is no definitive, all-encpmpassing' statement of 
media selection criteria. 
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The difficulty of the task of media selection is 
increased by the proliferation of media choices 
available and the limited operational experience of 
most instructors and instructional designers. 



What we have outlined is an approach to media 
^ selection. There is no definitive answer to the 
question: Media — which one? There are, 
however, lots more questions, and asking those 
questions can insure more appropriate selection of 
media for your instructional programs. 
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TIEING IT TOGETHER: TRENDS IN EDUCATIONAL 
COMMUNICATIONS AND TECHNOLOGY* 

WiUiam L, MiUard, Ed.D. 
School of Medicine, University of Southern California, Los Angeles 



As educators, we are concerned with educational 
technology as it relates to the development and 
implementation of comprehensive programs in 
veterinary medical education. When we speak 
about educational technology, we are talking 
about .the management of information, the 
technological devices and systems used in storing, 
retrieving and displaying information; and the 
personnel and financial support required to 
Implement our educational programs. 

Educational technology is in a continuous process 
of change related to developing new educational 
objectives, improving instructional practice, and 
organizing and implementing curricuia. 

Our application of educational technology to the 
teaching and learning process should place 
emphasis on the learner and his achievement. We 
are concerned with improving the conditions of 
learning, especially the individualization of 
learning and the ultimate change in the learner. 

This changing educational technology suggests 
several guidelines which we may apply in planning 
for teaching and learnmg activities and selecting 
instructional resources: 

1. Improvements in education and training 
have generally been more closely associated 
with innovations in educational procedures 
and techniques than with technical devices 



and systems. Independent study, 
individualized learning programs, 
programmed instruction, team teaching, and 
small-group interaction are a few of these 
educational procedures. 

2. In general the effectiveness of an 
instructional presentation depends primarily 
on the learning message designed rather than 
on the medium employed in the 
communication process. The inherent 
"hardware" characteristics of media have 
never been the primary factor in the 
improvement of the effectiveness of 
instruction. 

Technological devices and systems 
frequently make possible educational 
changes that have long-range and 
far-reaching • effects, but these changes 
depend upon the manner in which the 
tpjhnology is employed and the purposes 
vy/hich it serves. Consequently, technological 
devices and systems should be looked upon 
as the means of implementing new,- 
educational procedures and techniques. 

3. Educational technology has the potential of 
handling most of the routine instructional 
tasks usually performed by the instructor. It 
can provide the means of assisting the 
educator in organizing and handling the vast 



* Condensed version of a multimedia presentation given at the 5th Symposium on Veterinary Medical 
Education, Athens, Georgia. 
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amount of dat^ he derives from and about 
the learner. It can provide the means of 
efficiently ennploying the vast store of 
information, educational materials, and 
programs which will increasingly become 
available to the educator in 'ireating, 
programming, and evaluating learning 
experiences. 

4. Identification of the needs, interests, and 
concerns 'of the learner are essential in the 
development and implementation of learning 
programs. Stress is placed on the kind of 
behavior exhibited by the learner following 
instruction and the development of suitable 
means for increasing feedback and 
evaluation of learning outcomes. 

5. Individuals nnust be geared to accommodate 
to rapid changes in educational programs, 
instructional methods, and technological 
resources. New approaches to teaching and 
learning require different activities and 
functions on the part of all 
concerned — teacher, student, media 
personnel, administrators, and technicians.' 
There is an increased division of labor in the 
design, preparation, and implementation of 
educational programs and learning activities. 
The traditional role of the teacher as 
presenter will change to^ the tole of 
developer and nnanager of learning 
experiences. The learner will become a nnore 
active participant in the . development, 
preparation, and evaluation of the learning 
programs he uses. 



Learning Activities 
Individualized Learning 

It has been estimated that individualized learning 
programs will ultimately account for a large 
percentage of a student's education and training in 
medicine and the health professions. This 
commitment will have profound effects upon our 
approach in planning future instructional/learning 
systems and facilities, plus the required media 
support resources and services. 



These individualized (earning activities will not be 
restricted to specified on campus spaces but will 
occur in many different locations employing a 
variety of facilities and services, including remote 
learning centers, homes, vehicles, and nearly any p 
jocation the learner may choose. 

Individualized learning activities will frequently be 
under computer management providing a capacity 
to interact with the student and prescribe learning 
experiences tailored to the specific learning 
characteristics and needs of tho student. 

We may anticipate increased miniaturization of 
instructional devices with' a capacity to store, 
retrieve,' and display a wide range of learning 
resources. These devices will provide for 
interactive student response, self-testing, 
evaluation of performance, and feedback of the 
data to a centralized total information system for 
automatic record keeping and student or 
instructor retrieval for analysis, review, and 
counseling purposes. 



Group Learning 

Long-range emphasis on individualized learning 
suggests a reduction in the need to bring large 
groups of students tdg^etfier at one time in one 
location for instructional purposes. Although large 
group instruction will continue to play an 
important role in the overall educational program, 
greater selectivity will be applied as to when, 
where, and under what conditions this 
instructional modality will be employed in the 
teaching/learning process. 

Small-group learning activities will continue to 
play a vital role in the synthesis and integration of 
learning provided through setf-instruction and 
large group presentation. 

Both large- and small-group instruction will require 
direct access to a total information system for 
on-demand display and recording of relevant 
information and learning materials. These 
instructional modalities will also require a 
compiehensive media software and hardware 
suppoi I system. 
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Learning Materials 

The preparation of learning programs will require, 
initially, the use of existing media formats such as 
film, television, slides, and audiotapes. However, 
technological innovations will gradually reduce the 
need for a multiplicity of separate media by 
providing a combination of electronic-microform 
system for storing and retrieving both still and 
motion material with sound. 

The trend toward miniatruization of both software 
and hardware consistently reduces space 
requirements for housing production resources. 
Miniaturization also increases the ease with which 
information can be recorded and relieved by 
many individuals under nearly any conuition and 
location. 

The classification, storage, retrieval, and 
distribution of textual, audio, visual, and data 
resources will be handled by a total information 
system. Such a system will serve the functions now 
provided by the traditinnal library and 
instructional materials distribution center. It will 
also provide the techniques and devices needed by 
the instructional team or teacher to manipulate, 
organize and structure these "banked" resources 
into appropriate lessons or courses of study. 



Facilities and Spaces 

Educational technology is constantly ^hanging 
with the result that new facilities will be required 
in different space configurations. Curriculum 
changes, new, teaching methodologies, and new 
media equipment must be accommodated by new 
designs, instrumentation, and utilization of 
building space. Although many of these changes 
cannot yei be identified, they should be 
anticipated in future planning. 

1. Wherever possible, space should be planned 
so it is convertible from one use to another. 

2. Wherever possible, multi-use space should be 
provided if it can be designed to serve the 
specific functions well with a minimal 
amount of conflict. 



3. A variety of learning spaces should be 
provided, with special emphasis on small 
group and individualized learning facilities, 
with the latter being dispersed widely 
throughout the educational institution. 
These learning spaces should have direct 
access to the total information system plus 
built-in self-contained media systems for the 
display of leiTning materials. Adequate 
attention ."T^nst be given to environmental 
factors such as lighting, acoustics, and 
climate control. 

4. A variety of office, work, and production 
spaces will be required for personnel and the 
hardware and software of the media 
resources services. Certain of these functions 
will be housed in a centralized area. Other 
media support activities will require space 
dispersed throughout the educational 
institution. Most of these spaces must have 
direct access to the total information 
system. 



Personnel 

A comprehensive system of media resources and 
services will require a large staff of professional 
and technical personnel. The team concept of 
preparing learning programs and materials will 
substantially increase the ratio of educational 
support personnel to health professionals. 

The educational support team will bR comprised of 
a variety of media, evaluation, curriculum, 
engineering, electronics, computer, and 
information/communications professionals and 
technicians. 



Goals of Educational Technology 

The goalSfOf.e.ducational technology are to: 

1. Make available to teachers and others 
concerned with the development of learning 
resources and related instruciional 
presentations all media, technology, services, 
and systems required in the educational 
communications process. 
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2. Make available to. the learner all media, 
technology, services, and systems required in 
the instructional/learning process. 

3. Establish and make available suitable 
systems and methods for feedback and 
interchange between teacher and learner. 



Realization of these goals should in all probability 
place emphasis on the preparation and acquisition 
of instructional/learning materials related to three 
developmental phases: 

Phase I should focus on the expenditure of 
funds for the development of 
instructional/learning programs utilizing 
such commonly available software as books, 
film, slides, audiotapes, and related 
materials. Low cost hardware, such as slide • 
viewers, audio cassette players, and cartridge 
film devices should be employed as the 
primary means of student access to these 
learning programs. 



Phase II should focus on improving the 
accessibility of instructional/learning 
programs to students at a wide variety of 
locations through the use of such electronic 
media as closed-circuit television and dial 
access information retrieval systems. 

Phase III should focus on the automation of 
information acquisition, storage, retrieval, 
and distribution employing computer 
management and microform storage 
techniques. Interconnection with regional 
and national information networks will 
vastly increase the availability of learning 
resources for integration within the curricula 
of the medical center educational program. 

The degree of hardware sophistication to be 
implemented during each of these developmental 
steps should involve a cost-effectiveness analysis of 
various software/hardware systems combinations 
capable of supporting educational programs 
designed to achieve criterion levels of learner 
performance. 



64 



REFERENCES 



Banister, Richard E., "Low-Cost Technology in Higher Education/' Educational Technology, VIII Number 21, 
{November 15, 1968), 14-17. 

Deterline, William A., "Learning Theory, Teaching, and Instructional Technology," Audiovisual . 
Communications Review, 1965, 13, 407. 

Diamond, Robert M., "Large Group Instruction — Outdated: A Look at the 70's," Educational Technology, 
VIII Number 23 (December 15, 1968), 15-16. 

Gagne, Robert M., "Educational Technology as Technique," Educational Technology, VIII Number 21, 
(November 15, 1968),5~13. 

Heinich, Robert, "What Is Instructional Technology," Audiovisual Instruction, Vol. 13, No. 3, (March 1968), 
220-222. 

Heinich, Robert, "Educational Technology as Technology," Educational Technology, 1968, 8:1, 4. 

Hoban, Charles, F., "Man, Ritual, The Establishment and Instructional Technology," Educational Technology, 
VIM Number 20, (October 30, 1968), 5-1 1. 

Kurland, Norman D., "The Impact of Technology on Education," Educational Technology, VIII Number 20, 
{October 30, 1968), 12-1 5. 

Lange, Phil C, "Technology, Learning and Instruction," Audiovisual Instruction, Vol. 13, No. 3, 
(March 1968), 226-231. 

Ullmer, Eden J., "The Meaning of Instructional Technology: An Operational Analysis/' Educational 
Technology, VHUNumber 23 (December 15, 1968), 10-14. 

/ 



65 



TESTING AND EVALUATION FOR THE SCIENCES 



WUIiam D, Hedges, Ph.D. 

Chairman, Department of Elementary Education 
University of Florida, Gainesville 



Historical Stages of Test Development 

Man's attempts to measure what students learn in ^. 
'school' may be roughly classified into four major 
periods. The first period dates from antiquity to 
about 1800 A. D. The second period encompasses 
approximately the nineteenth century and 
includes the development of statistical techniques 
which were to make possible the standardized 
testing movement. The third period dates from 
about 1910 to about 1960. At this writing, there is 
evidence of a fourth era in testing and evaluation 
which may result in the development of new 
instruments in testing and new techniques in 
evaluation. 

When the first formal written examination first 
made its appearance is unknown, but it is known 
that China had developed a complex system of 
national examinations at least as early as 
2200 B.C.I 

Perhaps the earliest record of an ora/ exanjination 
is that found in the Old Testament 
(Judges, 12:5-7). Scheidemann^ describes the 
testing of the Ephraimites by the Gileaditesat the 
passages of Jordan. If the Ephraimite could not 
pronounce the word "Shibboleth," but instead 
called it "Sibboleth," he was slain. Today, 
doubtless we would call this a criterion referenced 
test or test item. 

There are numerous recorded instances of variety 
of performance tests administered by primitive 
tribal groups to youths as part of the initiation 
rites in preparation for entry upon adult status. 



These performance tests, among other things, 
required the youth to withstand hunger and thirst, 
tested his bravery and measured his skill in hunting 
and fighting.3 

Records exist of early objective standards for 
comparing attainments of youth in Greece. Thus, 
"The test of the development of a Spartan boy 
was whether or not he could endure hunger, cold, 
and exertion; the test of the Athenian boy in 
determining whether he could run, jump, race, 
declaim, repeat literature, and debate."^ 

The written university examination apparently was 
nnt used by the universities until less than 
oUO years ago. "Probably the first written 
examination at a university was in 1702, when it 
was introduced at Cambridge, England.''^ 

In short, man has recognized '^differences between 
individuals for thousands of years and accordingly 
has subjected them to numerous performance, 
oral, and written tests. As recently as the 
beginning of the nineteenth century, however, the 
instruments available for such measurement were 
crude and unreliable and did not provide very well- 
for comparative purposes. It is the appearance of 
this comparative bit that has really gotten us into 
some silly difficulties as we will see. 

"The first examination of historic interest, in our 
own country, is the Boston examination of 
1 845."6 This was a comprehensive written 
examination made up of 154 printed questions in 
such subjects as natural philosophy, geography, 
history, and astronomy. It was administered to 



67 



some 530 fourteen year old students in 
twenty-three elementary schools in Boston. The 
need for such an examination arose from the 
increasing difficulty for the Boston school 
committee to orally examine all the pupils as 
enrollments increased. 

The significance of the Boston examination lies in 
its impact upon Horace Mann, the American 
educator who had a great deal to do with the 
shaping of American education. Mann said:'"*'We 
venture to predict that the mode of examination 
by printed questions and written answers will 
constitute a new era in the history of our 
schools. "7 How prophetic he was! How 'good' the 
new era was to be is another question. 

The earliest record of any form of standardized 
measurement of the results of teaching is less than 
one hundred years old. Professor E. L. Thorndike 
reported the development of a scale book for 
handwriting devised by an English school master, 
the Rev. George Fisher, of the Greenwich Hospital 
School in 1864.8 

In our country the first standardized test of which 
there was recoiu was developed thirty years later 
in 1894 by J. M. Rice.9 Dr. Rice conceived the 
idea of trying to settle the conflicting claims of 
school men about the relative merits of various 
courses of study and study methods. As his first 
subject, he chose spelling and developed a fifty 
word spelling test. But he was ahead of his time. It 
is evident that, as late as 1900 the standardized 
test did not exist 

As the nineteenth century drew to a close, some 
developments outside of education were to make 
possible rapid growth in the field of educational 
measurement shortly after the turn of the century. 
Of major significance was the development of 
statistical methods by Galton. Another significant 
development which just preceded the interest in 
and development of standardized achievement 
tests was the research on mentality and the 
development of psychological testing. 

Thus, the stage was set around 1900 for the third 
. period of growth in testing. The most prominent 



leader in this education measurement movement 
after 1900 was E. L. Thorndike who published his 
first book on mental and social measurement in 
1904. Others quickly followed. • 

By World War I, the basic statistical techniques 
had been developed and the need clearly 
demonstrated for the production of better 
measurement of what schools were accomplishing 
with their students. By the 1920's so many tests 
had been produced in so many subject areas that 
Monroe published a small bibliography of 
standardized tests for secondary schools, By 1920, 
the "battle" for educationaf measurement by 
means of objective tests had been won. This does 
not mean the tests were very good because they 
were not. 

But, subsequent to 1920 there developed a 
tremendous emphasis on the quality of test 
construction — and we are still struggling with this 
one, i.e. how to improve our tests. 

A new direction developed in the thirties in 
educational measurement influenced by 
Ralph W. Tyler with his work with the Eight-Year 
Study. Tyler emphasized the concept of evaluation 
as contrasted with testing and was concerned with 
". . . the measurement of a broader set of 
objectives (i.e., more aspects of a person's 
characteristics and response tendencies) than the 
ordinary achievement tests deal with. "10 

The testing movement had a number of influences 
on the curriculum, one of which was that if a 
teacher is to construct a good test, he must be 
aware of what he is trying to measure, i.e. aware of 
his subjectives. At least, he should be aware of his 
objectives. The objectives needed to be specific 
and beftavioristic instead of vague and glittering 
generalities. 

Since World War II, we may characterize the 
movement until relatively recently as the period of 
test batteries and testing programs." 

One of the most significant more recent 
developments in the field of educational 
measurement was the publication of Bloom's The 
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Taxonomy of Educational Objectives, 
Handbook I; Cognitive Domain. The function of 
this taxonomy was to provide a system whereby 
test items could be classified according to degree 
of complexity of mental processes required to 
answer the question. 

In short, the half century from 1910 through 1960 
witnessed a tremendous explosion in the field of 
standardized and teacher-made testing. In 
particular, standardized tests in science and 
virtually every other field reached a high point in 
efficiency insofar as what they measured was 
concerned, but they incorporated a tremendously 
wasteful idea — and still do. 

Permeating most of the standardized tests and 
spilling off into the teacher-made test was the 
concept of selection instead of development. The 
purpose of evaluation became and remains 
primarily the grading and pidgeon-holing of 
students in order to 'select out of the system' the 
unfit. The effects on students at all levels have 
been devastating. On the one hand there were few - 
saying tests should help diagnose the needs of 
these persons to help them learn while others were 
saying the function was, on a comparative basis, to 
get rid of these students because they were the 
lower achievers at a given point in time. 

When students are evaluated on a normative or 
curve basis, this places them at swords points with 
each other, guarantees failure for many aside from 
any criterion of excellence and cultivates the 
tendency for students to avoid cooperating with 
each other. 

"As testing and other forms of evaluation are 
commonly used in the schools, they contribute 
little to the improvement of teaching and learning, 
and they rarely serve to ensure that all (or almost 
all) learn what the school system regards as the 
important tasks and goals of 'the education 
process."! 1 

A much broader view of evaluation is emerging 
today, well described by Bloom, Hastings and 
Madaus.l2 They. see evaluation as encompassing: 
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"Evaluation as a method of acquiring and 
processing the evidence needed to improve 
the student's learning and the teaching. 

"Evaluation as including a great variety of 
evidence beyond the usual final paper and 
pencil examination. 

"Evaluation as an aid in clarifying the 
significant goals and objectives of education 
and as a process for determining the extent 
to which students are developing in these 
desired ways. 

"Evaluation as a system of quality control in 
which it may be determined at each step in 
the teaching-learning process whether the 
process is effective or not, and if not, what 
changes must be made to ensure its 
effectiveness before it is too late. 

"Final evaluation as a tool in education 
practice for ascertaining whether alternative 
procedures are equally effective or not in 
achieving a set 5f educational ends." 

Little doubt remains today amorig'educators that 
effective teaching and learning include testing and 
evaluation. Thus, that student's study habits are 
modified by the instructor's testing emphasis, 
either for good or for ill, is well known to most of 
us. This factor is evidenced in the tendency of 
students to ascertain how each instructor tests in 
order that they may know how to prepare j^or his 
tests. 

If student's study habits are affected by the 
instructor's testing proclivities, it follows that if 
the instructor emphasized trivia, the students will 
tend to study for trivia. Furthermore, to"teach 
without some sort of feedback on the outcome 
seems to us today to be virtually unthinkable, 
Feedback is important for students themselves, for 
the teachers in looking at their own effectiveness, 
for the teachers in making changes in their 
teaching strategies and for curriculum makers in 
revising and updating their programs. 



Unfortunately, considerable confusion has grown 
up about the efficacy of testing at all levels from 
kindergarten through the graduate school. This 
confusion derives, at least in part from ambiguity 
as to the purposes of testing. It derives too from 
confusion between the very discretely different 
processes of measurement and evaluation. Scores 
earned on a valid and reliable test (measurement) 
are one thing, while the subjective judgements the 
teacher brings to bear on the meaning of these 
scores as they relate to the individual student, his 
own teaching, etc. (evaluation) are another. 

The purposes of this paper are to point up some of 
the more crucial concepts which might be kept in 
mind by veterinary medical educators as they 
devise, construct and refine their testing programs 
in their various colleges. More specifically, this 
paper emphasizes how these crucial concepts relate 
to the emerging emphasis on mastery learning. 

Specific Functions of Testing 

Of primary importance is clarity on the functions 
tests are to perform. While these decisions rest 
primarily with the curriculum makers and with the 
instructors, at the vert least these persons should 
be clear about the functions their tests are 
designed to accomplish. A number of the more 
major functions available as options are briefly 
presented below. 

First, when test items or other testing measures are 
clustered around clearly defined and essential 
objectives, i.e. around the more significant 
elements of the curriculum, students tend to 
concentrate their learning efforts towards these 
objectives. 

Second, careful pre-testing can help the student 
and the instructor know where and how to begin. 
Obviously this assist is particularly crucial for 
young children, but it certainly is helpful in truly 
individualizing programs in higher education. 
Typically, however, we begin 'at the beginning,' 
i.e. by assuming an equal state of ignorance (or 
knowledge) among all students. 



Third, post-testing can document mastery or the 
lack of it, providing a basis for assessing such 
factors as teaching effectiveness, student 
achievement and curriculum modification. When 
coupled with pre-testing, indications of growtii or 
ciiange in~students can be detected. Not to bs 
overlooked is the guidance for additional learning 
that can be gleaned from effective post-testing 
procedures. 

Fourth, interim or formative testing serves a 
monitoring function enabling both student and 
teacher together to. adjust their efforts to 
maximize their effectiveness. Such frequent 
mo nitoring contrasts with more infrequent 
summative testing for 'grades' which may serve 
primarily to induce tension in some students. 
There is evidence that little or no real motivation 
results under conditions of considerable anxiety. 

Fifth, the development of tests or reloted criterion 
measures tends to sharpen and refine the 
professor's conception of his goals for instruction. 
More specifically, if the instructor makes up his 
objectives first, builds his tests second, and onlv 
then develops his materials and teaching strategies, 
he drastically increases his efficiency as well as 
insuring balance of the program. Contrast the 
above approach with the more traditional selection 
of materials first more or less haphazard choice of 
test items second, and then, when asked, the more 
or less 'thinking up' of the objectives! If we know 
anything about teaching, we know that a sharp 
focus on that which we are attempting maximizes 
our effectiveness. 

Sixth, the learning function itself served by 
frequent testing should not be overlookeo or 
minimized. Learning theorists maintain that recall 
enhances retention. The discovery of correct 
responses to carefuiiy constructed oh\GCtm test 
items goes beyond simple recognition and involved 
considerable recall. 

Throughout all' of the above, the concept of 
feedback is of extreme import. A simple physical 
analogy should suffice to clarify this point. For 
example, the rifleman can only modify his aim for. 
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increased accuracy as he learns about the location 
of impact of his earlier shots. If we keep this point 
in mind as we plan and construct our tests and 
testing procedures we may avoid some of the more 
common pitfalls which have brought the testing, 
movement into disrepute. Let us examine some of 
these more common weaknesses in . our 
teacher-constructed tests. 



Common Weaknesses in Teacher-Constructed 
Paper-and-Pencil Tests 

One major weakness is the prevalence of testing 
for low-level primarily memoriter type learning. A 
study by the author and several colleagues of the 
mid-year examinations in physics, . chemistry, 
biology and 9th grade science in 100 Virginia high 
schools revealed the predominant emphasis to be 
the simple recall of information. 13 

In more specific terms 78 percent of the test items 
were concerned with the recall of factual 
information (level 1.00 of Bloom). Of this 
78 percent, 64 percent of the items (or 50 percent 
of the approximately 14,000 questions) required 
only the simplest form of recall (level 1.10 of 
Bloom's Taxonomy). If it is assumed that teachers 
tend to place on their mid-year examinations those 
things which they consider most important and if 
it is further assumed, as mentioned earlier, that 
students' study habits are affected by the teacher's 
testing habits, then the status of science teaching 
in the secondary schools of Virginia at that point 
in time was no less than alarming. Studies similar 
to the above in the various schools and colleges of 
our institutions of higher learning might reveal 
whetlier there are similarities to this state of 
affairs 

A seconJ major weakness in our 
teacher-constructed tests consists of a lack of 
awareness by the instructor of the major strengths, 
limitations and functions of the different types of 
tests, test questions and other testing instruments 
and procedures. In the preparation of 
paper-and-pencil instruments there is a lack of 
awareness of the specific functions of the essay, 
multiple-choice, true-false, matching, completion, 
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problem solving and situational or application type 
items. Mindless debates abound among faculty 
about whether or not to use such and such a 
type — missing the point i.e. that the criterion of 
use should hinge upon the instructor's purpose and 
whether or not a particular type of question 
enables that purpose. A number of references for 
the interested reader is suggested in an attached 
bibliography. 

A third major weakness is lack of basic skills in the 
principles of item writing and of test building in 
general. Planning, preparing, administering and 
analyzing tests requires some care and skill. An 
excellent free resource available for the asking is 
.;put out by Educational Testing Service entitled 
.'Tests and Measurement Kit, Princeton, New 
Jersey, 08540. ' 

Fourth, simple item analysis procedures designed 
to build pools of effective, discriminating test 
items clustered about specific objectives are too 
rarely employed. There is a paucity of skill in the 

- elementary statistical techniques useful in 

analyfjtig and improving tests. Coupled with this 
limited use is a general lack of understanding of 
basic concepts in measurejnent such as validity, 
reliability, correlation, standard error, and the like. 

Still another major weakness derives not as much 
from weaknesses in the tests themselves but in the 
use of the results. One of the major and most 
prevalent 'curses' of the testing movement has 
been the blind application of the normative 
concept to the summative evaluation of students. 
Under the typical approach to grading, the normal 
curve concept is applied which means, by 
definition, that half of^ the students in each class 
will be assessed^as doing poorly no matter their 
level of achievement: Thc^ converse is also true. 
Also, mindless application of this normative 
concept has, over the years,, aroused much fear, 
tension and hostility in students, while forcing 
intense competition aside from any particular 
standards of excellence. The goal has been to do 
better than the others, whatever that is, and the 
evidence is that the variance in quality among 
institutions, among colleges within institutions and 
among departments within colleges, not to 
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mention among faculty within a department has 
been considerable. . 

If, on the other hand, instructors were to state 
with precision that which their students are to 
learn and then to determine whether or not such 
learning is attained, success becomes defined as 
reaching the level of mastery established for the 
objective prescribed and becomes independent of 
what the student's peers do or do not accomplish. 
This is a major concept in mastery learning as set 
forth in the work of Block, Bloom and others so 
more need not be said about it here. 

There is evidence too, as mentioned earlier and as 
Cronbach, Bloom, Block and others have 
indicated, that the very clear precise specifications 
of that which is to be learned in behavioral, 
observable terms at a given level of proficiency 
under clearly stated conditions is about hal? of the 
battle. In such circumstances, tests tend to move 
out of the guessing game category and truly 
become assists for student and teacher alike. A 
major source of suspicion and distrust is removed 
for the student no longer has to 'psyche out' the 
teacher. 

Unfortunately, at this stage another problem tends 
to emerge. This is that the students begin to do 
too well, i.e. there are too many A's which may 
bring down on us the wrath of the administration. 
This is interesting, if not ironic when it is 
remembered that presumably at least one of the 
functions of teaching is learning and one of the 
indictments of effective teaching becomes the 
accusation that too many of the students are 
learning too well! But that such is the case 
becomes all too clear to that faculty member who, 
having informed his students of exactly what they 
must master to a prescribed standard of excellence 
finds himself forced to turn in 27 A's and three B's 
in a class of 30. One way out of this, of course, in 
recent years has been to go to a pass-incomplete 
grading system. 

While the tenor of this paper to this point has 
i stressed the use of pencil and paper instruments, 
the intent is not to suggest that these are the only 
or even necessarily the most helpful instruments. 
Increasingly there are research efforts to relate 



these paper-and-pencil devices to situations 
involving observable behaviors so that one or the 
other may be used interchangeabh' 14 
Additionally, there are many elements ui a 
program which, to ^ate,. can- only be Fjsessed 
through direct observation of the student's 
performance in situations. This.' leads' us to^^thj. 
question of what to measure. 



What to Measure 

You will hear shortly from 
Drs. Smith and Gardner regarding the cognitive 
and affective domains, i.e. the mental- and the 
emotional dimensions of learnmg so 1 will' only 
briefly refer to them here. In essence, we tend to 
stress the cognitive dimensions of learning and 
slight the attitudinal. And yet, you and I know 
that success of most of us in our chosen fields 
depends as much on our feelings, beliefs, opinions, 
values, attitudes and skills in human relations as on 
our technical expertise. Who ever was fired 
because "he did not know" in contrast to "he 
could not work with anyone—even himself?" I 
just want to say that progress is being made in 
assessing the affective domain despite the cultural 
reluctance in America to stepping into what is 
commonly assumed to be one's own private 
business. 

Not only must 'what, to measure' move over into 
and include -dimensions of the affective domain 
but, increasingly we must raise the level of 
measurement in the cognitive domain from the 
heavily memoriter or recall to increasingly insist 
upon ability to apply, analyze, synthesize and 
evaluate as suggested by Bloom and others. 

'J 

Improving One's Testing Program 

A number of alternatives exist in considering how 
to improve one's program of testing and 
evaluation. Basic to all such attempts, however, is 
the necessity for faculty to learn how to write 
objectives clearly and well and for faculty to have 
a few houi-s of basic instruction in the principles of 
testing and evaluation. 
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A sound next step then is for assessment 
procedures to be developed including pools of test 
items clustered about objectives. With some 
cooperation and coordination among professors at 
various institutions working in similar subject 
areas, each professor would not need to rediscover 
America. 

It has been possible for some time now for the 
statistical analysis of our test items to be handled 
through a computer including item analysis, 
reliability and validity. 

Increasingly, the generation of the tests 
themselves, particularly the paper-and-pencil 
variety, is accomplished through item pools as 
needed. Each faculty seriously interested in 
improving its testing program needs a person 
qualified in these areas to organize and implement 
it in collaboration with the faculty. 



But, and this is probably the major point of 'where 
we might go.' The modern evaluation program 
stresses formative evaluation procedures, 
i.e. systematic evaluation dunng the processes of 
curriculum development, of teaching and of 
learning in contrast to only suwmative evaluation 
at the end of a course or term or program. In 
summative evaluation, you afterwards and 
not during. This is a crucial difference. It changes 
the entire concept of evaluation from one of 
selecting out to one of enabling a person to figure 
out how to stay in. It becomes a rewarding, 
helpful process instead of a punitive negative one. 

In summary, I have endeavored to briefly trace the 
history and major trends of evaluation over, the 
past few years. Certainly, it is a sketchy attempt in 
the brief space allowable, but hopefully you have 
at least a fuzzy idea of whence we have come and 
where we are. 
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INTRODUCTION 

At this stage of the symposium participants should 
have developed their minicourse. Topics have been 
chosen, objectives have been staled, and media for 
presentation have been selected. The final stage is 
the conversion oi objectives to evaluation 
situations. My task for this session of the 
symposium Is to deal with the cognitive evaluation 
of students with an emphasis. on what's new in 
evaluation. As a starling point I would like to 
discuss briefly what I will label the "traditional" 
approach to student evaluation. This traditional or 
usual approach for testing students will then 
become my "straw man" for the remainder of the 
paper. 

The first characteristic of the traditional approach 
is the behavior students exhibit while preparing for 
course examinations. Preparation often becomes a 
guessing game students play to "psyche out" the 
instructor in an attempt to find out what is 
important. 

The second sign of the traditional approach is a 
preponderence of test JieiDs.. that ask for 
memorizing specific facts and terminology and the 
avoidance of items testing comprehensiori and 
application. 

Yet another characteristic is an emphasis on 
spreading student's test scores so that A's, B's, C's, 
D's and E's can be easily assigned. This is often 
accomplished by asking increasingly pedantic 
questions taken from isolated bits of information 
used as filler for examination purposes. In other 
words, an attempt is made to distribute students 
on some approximation of a normal curve. . 



Finally, the traditional approach is characterised 
by the use of tests as unidirectional assessment 
devices administered at the end of the learning 
situation. By unidirecrionaiity I mean that tc:sts 
are usually given by instructors to assess student 
competency with little or no thought given to the 
-evaluation of the instructional process and 
teaching effectiveness. Seldom is the assessment 
process used to give feedback for the improvement 
of instruction. 



CRITERION-REFERENCED TESTING 

In contrast to the traditional approach to 
assessment is what has been termed 
"criterion-referenced testing." This approach 
attempts to indicata with as much precision as 
possible; the degree to which each student has 
. mastered the learning objectives specified in a 
course or unit of instruction. Criterion-referenced 
testing is, of course, a direct consequence of the 
learning by object'*. es approach. 



From Objectives to Evaluation 

The process of measurement is secondary to 
•that of defining objectives. The ends to be 
achieved must first be formulated. Then' ^. 
measurement procedures can be sought as 
tools for appraising the extent to which 
those ends have been achieved.* 

The ability to contruct meamngiul and valid 
assessment devices is to a large degree dependent 
upon how well performance objectives have been 
specified. Well written objectives, which spell-out 



*Nunnally, J. C, Educational Measurement and Evaluation, New York: McGraw-Hill, 1964. 



in an explicit manner the behavior you wish the 
student to demonstrate, go a long way in assuring 
the construction of useful testing instruments. 
Vague and nebulous objectives make test 
construction difficult and leavn the student 
tjuessinn as to what is expected of him. Recently 
an instructor was asked to specify the objectives of 
his' course in preventive medicme. The following 
obiectivfis were given in reply: 

1. Create conditions which develop in ihe 
student the comprehension that an holisiic 
concept of disease is essential for 
understanding and correcting problems, 

2. Emphasise factual information related to the 
environment as it applies to alteration or 
effect on the host. 

3. Demonstrate the utilization of facts and 
principles to the understanding and solving 
of the problems of community health in 
lower animals and man. 

These three examples are too vague and general to 
be considered useful learning objectives. Even 
though they can be viewed as goals upon which 
further planning might be based, they cannot be 
tested directly and therefore (do not make very 
useful objectives. You cannot directly test a 
student to determine if he has a "holistic concept 
of disease" or if he can "utilize facts and 
principles/' unless these states are defined 
operationally. What is needed is a number of direct 



behavioral learning objectives and cuiesiiofis which 
collectively can be used to reliabiy inlei wfiethcr 
Of not the student has the above mentioned 
capabilities. 

Contrast this with the lollowintj objective: 

Objective: The student should be able to 
recognize the physiological functions and 
associated organs that may be involved in 
loss of control as a result of injury to the 
sacral vertzibrae, 

This instructional oojeciive is stated precisely and 
specifies what the learner must be able to DO 
when he is demonstrating his mastery of the 
objective. When objectives are stated in this 
manner, the construction of test items becomes a 
relatively easy, task. 

Question: Injury to -the sacral vertebrae 
often results in loss of control 

of which [s ' 

an added danger to the animal's recovery. 

a. Bladder and kidney function 

*b. Bladder and bowel function 

c. Bowel and stomach function 

d. Most motor function in rear limbs 

e. Most sensory response in rear limbs. 
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Dr. Dwight Bennett of Purdue is economical in writing objectives and test items in a second example: 



Objectives: 

I. Identification of Parasites 
Parasite 

Fasciola Hepatica 



Dicrocoelium dendriticum 

Fascioloides magna 
Paramphistomum cervi 



Be able to identify by 

a. whole parasite grossly 

b. whole parasite mounted on microscope slide 

c. section of parasite in liver (microscopic) 

d. damaged, cirrhotic liver (suspecl) 

a. whole parasite grossly 

b. whole parasite mounted on microscope slide 

c. section of parasite in liver (microscopic) 

a. whole parasite grossly 

a. whole parasite grossly 

b. whole parasite mounted on microscope slide 



Information Given 
Liver of ruminant 



Liver of ruminant 



Liver of ruminant 

Rumen 
OR 
Reticulum 
OR 

feces from ruminant 



Questions: 

A. Identification 
dpt.) 1. From abomasum of a calf 
Genus 



dpt.) 2. From small intestine of a lamb 
Genus 



(1 ptj 3. From liver of a sheep 
Scientific name 



The above instructional objectives are s ated precisely and specify what the learner must be able to 
DO when he is demonstrating. his mastery of objectives. Too much cannot be said for the importance of good 
learning objectives for the. valid assessment of students' knowledge. When used properly they can aid the 
instructor in teaching, make clearer to the student what is expected of him and have exams more accurate 
measures of student learnmg. 
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Testing Higher Level Cognitive Outcomes 

If instructors were to survey their examinations, 
many would find that the items included in their 
tests are predominantly recall questions with few 
of the items aimed at testing comprehension and 
application. True, the study of medicine dictates 
thai a great deal of the student's time be spent in 
learning specific facts and principles, but this 
should not exclude the instruction and assessment 
of comprehension and application of medical 
principles. 

The brochure explaining the Symposium lists as 
one of \\9 objectives the following: 

Objective: Participants will be able to /ist 
the three criteria for an educational 
objective as described in Preparing 
Instructional Objectives by R. S. Mager. 

A possible test item for evaluating his objective 
might look like this: 

Question: List the three criteria for an 
educational objective as described by Mager. 

1. • 



3. 



It is possible that participants may be able to//sf, 
recall, or underline the criteria for an educational 
objective and still not be able to understand or 
apply the concept of an educational objective: 
Hopefully, individuals taking part in the 
symposium would, upon completion, be able to 
comprehend • the criteria for an educational 
objective and put their knowledge to work in 



situations that are new to them. If this is the case, 
then objectives and test questions should reflect 
the need for understanding and application of 
performance objectives. 

Objective (Comprehension): Given a list of 
performanc^" objectives, participants will be 
able to fdef tify educational objectives which 
meet the three criteria described in Mager s 
book (a 90 percent level of proficiency is 
required). 

Question (Comprehension): Which of the 
following educatio-ial objectives meet a// 
three of the criteria described in Mager's 
book? 

1. Given a set of 10 or 15 small numbers (8, 
14, 9 etc.). the student should be able to 
calculate, by hand, the mean or average 
for the set of scores. *No errors wilt be 
allowed and^j an^er will be rounded 
to two decimal places, 

2. Without the aid of a flotation device the 
student will swim the side stroke for 200 
yards in less than 10 minutes. 

3. me student should be able to understand 
the concept of homeostasis, f^roficiency is 
defined as getting 80 percent on an 
objective test, 

20. Given an adult human, skull the student 
should be able to name and locate the 
eight bones of the cranium and the 
14 bones of the face with no errors (two 
spelling errors will be allowed). 

The above objective and question demand more 
than simple recall of facts. Participants must be 
able to show an understanding of the concepts of ., 
educational objectives by identifying or 
recognizing objectives that meet the criteria 
specified in Mager's book. 

Objective (Application): Given that a 
minicourse topic has been selected, 
. participants will be able to demonstrate their 
knowledge of performance objectives by 
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constructing objectives for that minicourse 
which meet the thiee criteria described in 
Mager's book. This should be completed 
before entering the morning session on 
June 20th. 

Question (Application): Before entering the 
morning session on June 20th, write below 
your minicourse topic the objectives you 
have developed for it. Submit them at the 
door for appraisal. 

Here the participant is expected to demonstrate 
his knowledge of performance objectives by 
applying these concepts on some meaningful task. 
At this point, one is reminded of the last line of 
the Chineses Proverb printed on the brochure, "I 
do = I understand." 

Warning should be given that the tendency to 
write lower level cognitive test questions may also 
carry over while attempting to write learning 
objectives. Because test questions are derived from 
learning objectives, care snould be taken to avoid 
writing objectives that demand only simple 
memorizing and recall. 



Spreading Students (Why not to) 

Many teachers are happy when, at the end of the 
quarter or semester, students' cumulative test 
scores are extended over a wide range. They are 
even happier when so-called "natural breaking 
points" appear in the distribution. With a wide 
range of scores accompanied by "natural breaking 
points," the instructor can assign grades with 
relative ease and feel confident that he has fairly 
evaluated students' performance in the course. 
This attitude has been supported by traditional 
testing theory which, says that a good examination 
is one that contains questions having "item 
difficulties" (perrerifage of examinees who miss 
the item) of approximately 50 percent. 

With the concepts of learning for mastery and 
cri^erion-referencedjesting, the emphasis changes 
from one of spreading students along some 
continuum to one of assuring that each student has 
mastered the material. Each question attempts to 
t 3t a relevant concept and students are expected 



to exhibit proficiency on one unit of instruction 
before going on to the next. This often means that 
students are required to be examined more than 
once on a particular unit. A second, third or even 
fourth trial may be needed for some students to 
learn the material well enough to show mastery on 
the quiz or test. 



Formative vs. Summative Evaluation 

Traditional assessment of students usually occurs 
two or three times during a period of instruction 
with the primary purpose being the assignment of 
grades. This final or summative evaluation of 
student progress can be contrasted with what has 
been termed "formative evaluation." Formative 
evaluation is an ongoing assessment process in 
which evaluation instruments are applied 
throughout the course of instruction. Results are 
used as immediate feedback to both the student 
and the instructor as to how well they are doing. 
Students obtain information as to how well they 
are progressing on the learning objectives and 
teachers can assess whether instruction has been 
properly designed and conducted. Continual 
feedback to the student is especially important for 
those learning situations where successful mastery 
of the material at one level is to some degree 
dependent upon mastery of previous material at 
some lower level. In terms of the "minicourse 
concept," this means successful completion of one 
unit of study before progressing to the next. 

A final grade of B or C gives little or no useful 
information to the student as to his strengths and 
weaknesses, whereas immediate feedback or 
knowledge of results as to how well he is 
progressing on the objectives, has the effect of not 
only facilitating learning but of motivating the 
learner. 

Educators have long maintained that how well a 
student does on an* examination is as much a 
reflection on the quality of instruction as it fs on 
the student's knowledge of the material. With 
formative evaluation the instructor is allowed 
greater latitude in measuring the effectiveness of 
his program. Was the course and its objectives 
made clear? Did the various components of the 
instructional unit adequately convey the 
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knowledge that was intended? These are questions 
that can be answered using formative or ongoing 
evaluation, 

^'^'''^^ 

Summary of Criterion Referenced Testing 

The first and by far the most important aspect of 
criterion referenced testing is that each test item 
should be directly associated with a specific 
objective. The better the o'ljective the easier it 
becomes to write acceptable items. The second 
concern discussed was the need to construct items 
that test more than simple memorization of 
specific terminology and facts. Examinations 
should contain questions that test the higher order 
objectives of comprehension and application. 
Next, the purpose of teaching is to transcend the 
normal distribution. The instructor's ultimate goal 
should be to have each student eventually show 
mastery of the specified objectives by passing all 
or nearly all of the test items used to infer 
mastery. 

Finally, assessment should be an ongoing process 
which allows for continual feedback to both the 
student and the instructor as to how they are 
doing. In terms of the minicourse concept, 
evaluation should take place immediately 
following the unit of study or better yet, he 
testing activity can be built into the instruction 
itself. 



PLANNING AND CONSTRUCTING 
EXAMINATIONS 

The aim of assessment is to construct evaluation 
instruments that determine whether students have 
sufficiently mastered the material. Weaknesses in 
tests can be grouped into two broad categories: 

1. Students may have achieved the objective 
but the test questions may not measure that 
mastery. 

2. The student may answer the questions 
correctly, yet not have mastered the 
material. 



As an example of the latter weakness, without any 
knowledge of the subject matter, I was able to 
score 22 correct out of 25 true-false items on a 
core course exam by applying just one of the rules 
of true-false test construction. Well constructed 
items and good testing procedures help assure that 
students' knowledge is being validly assessed. 

Following are some of the concepts and guidelines 
use'i for constructing and evaluating classroom 
examinations. I have chosen to focus primarily on 
those problem areas that have come to my 
attention while working at the. Office of 
Veterinary Medical Education at Ohio State 
University. 



Validity and Reliability 

Any discussion of testing or test construction must 
incluor the concepts of validity and reliability. 

The concept of validity simply means "does the 
test measure what it is supposed to measure?" The 
primary concern of classroom achievement tests is 
"content validity." Does, the test measure the 
students' performance on some relevant set of 
tests. Unlike other types of validity that can be 
checked statistically, content validity is studied 
and judged by those who know" the content by 
"eyeballing" or checking the test items against the 
content supposed to be measured in a particular 
course of study. 

The best procedure for assuring content validity is 
to set up instructional objectives (discussed 
previously), and to outline the material presented 
in class. This allows the instructor to develop a 
"test plan" that covers fairly the material 
presented in class. If two weeks out of a five week 
test period is spent on a particular subject area or 
if a team member is given a certain block of class 
time, then a representative portion the exam 
should direct itself toward that particular area. 
One complaint often received ' from students 
concerning classroom tests is that considerable 
class time is spent m an area which is then only 
sparcely covered on the exam. The instructor 
should select items that representatively sarriple 
the content of material 'covered in class. 
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Reliability tells us how much confidence can be 
placed in our measuring device. It is an index or 
estimate of the ability of a test to measure 
performance consistently. !f a test is highly 
reliable then students would be expected to obtain 
similar scores on a similar test covering the same 
Content area. The primary concern of reliability 
salient to mastery testing is the length of the test. 
If in testing students on a particular subject we 
could ask all the possible questions covering all the 
material presented in the course we would have an 
exact index of how well each student had 
mastered the material. Because of lack of time and 
energy thir is not possible, so the instructor must 
select a sampling of possible test items to be given 
as the classroom exam. The importance of 
lengthening a test is that the more questions asked 
(assuming all items have content validity), the 
more adequate the sampling of all possible 
questions. By asking more questions the instructor 
comes closer and closer to estimating the student's 
true accomplishments or achievement. Instructors 
should take full advantage of the time given to 
testing to reliably assess the students' achievement. 
This is especially true for summative or final 
evaluation. Although a test should be as long as 
possible within the given time constraints, time 
should always be allowed for all or nearly all cf 
the students to complete the exam. 



Frequency of Examinations 

Formative assessment as an oiigoing process has 
already been discussed. It was pointed out that 
frequent formative evaluation . rovides feedback 
to the student as to whether hP '^as mastered the 
unit objectives or if he still has deficiencies to 
overcome. Quizzes can be given at the end of each 
unit, or self assessment questions can.be built into 
the unit much like programmed instruction. A 
combinat!:n of these two approaches is probably 
best for formative evaluation. 

The instructor is still faced with the problem o' 
assigning grades at the end of a period of 
instruction. Tliis summative evaluation is usually 
accomplished by administering one or more 
midterm examinations and a* final test. Scores 
derived from formative evaluation may or may not 
be combined with summative scores for the 



assignment of grades. My own feeling is that the 
quiz scores used to determine mastery should be 
included when assigning grades. 

A possible testing program might incorporate the 
following steps: 

1. A series of questions interspersed 
throughout the program to be used for 
feedback of student progress through the 
unit. These items would be for self 
assessment and should have no effect on the 
student's course grade. 

2. Upon completion of the unit, the student 
would be provided with a set of practice of 
self study questions. Students doing poorly 
on these questions may want to review parts 
of the unit, while a good score would signai 
readiness to take the unit mastery quiz. 
Again, these items would be for self 
assessment and would not affect grading. 



3. A short test or q' iz would be given at the 
completion of each unit with the results 
being used to infer unit mastery. If the 
student exhibits mastery, then he would be 
allowed to progress to the next unit of 
study. If, however, he failed to show 
mastery, he would be expected to return to 
the instructional program, correct the gaps 
in his knowledge and try the unit mastery 
quiz again. W-^^jn scores oh these unit 
mastery tests are to be used along with 
summative examinations for grading 
purposes, only those scores should be used 
where the student showed mastery. In other 
words, if the student failed the first quiz but 
learned the material and passed the second 
quiz, then the recorded score should be the 
second exam. 

4. Summative evaluation would be the final 
step in the testing program. This would 
usually involve the use of test items covering 
several units of study or final examinations 
covering all units in the course. The primary 
purpose of this kind of testing is the 
assignment of grades. 
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With the possible exception of the first step, all 
questions should be taken from the same universe 
of possible questions. Summative evaluation 
questions should be basically the same as ni'^stery 
quiz questions and both should resemble practice 
questions. 



Proficiency Levels for Mastery 

The ideal of instructional objectives and learning 
for mastery is that all the students learn all of the 
material and hence pass all of the items on the 
exam but for various reasons complete mastery is 
often difficult to^btain. Objectives may not be 
completely clear, the instructional program may 
be lacking or test items may not validly assess the 
students' knowledge; For these reasons, mastery 
on a quiz or examination should seldom require 
scores of 100 percent. This is especially true for 
the first few assessment efforts. But, as the 
instructor obtains feedback and is abie to improve 
his objectives, instrdctional program and 
assessment ^devices, test instruments will more 
validly" and reliably measure near the 100 percent 
mastery level and can be scored accordingly. 



Item Analysis and Feedback 

Recently, 1 aided an instructor in developing a pre 
and post test for assessing achievement in a 
continuing education short course. As was 
expected, all participants showed improvement 
from-the pre to the post test but an item analysis 
of the examination revealed that oh some of the 
items participants had shown a reverc^l in learning. 
That is, they had started the course knowing some 
of the basic connppts but had been confused 
during the course piesentation. It i^hes not take a 
great deal of insight to conclude that some aspects 
of the instructional program are iri need of 
improvement. 

Test item difficulty information can be used in at 
least three ways: 

1. Feedback for the improvement of the 
instructional program can be used to help 
pick out poor test items. What often 
happens is that when all or neErly^all of the 



students do exceptionally poorly on an item, 
it is not because all have equally mastered 
the material but that there are problems 
inherent within the question such as 
ambiguous wording. 

2. It can be used to point out those students- 
who are in need of remedial training. 

'3, It can give the instructor information 

concerning his attainment of instructional . ^ 
objectives. By looking at the different item ^ * 
difficulties an instructor could determine if 
the students were grasping the- concepts 
being tested. If ' they' were not, then he 
should possibly reiterate those concepts. 

It would probably be appropriate at this time to* 
•discuss the practice of using old test items. The 
available literature on this subject is scarce but 
most testing experts generally agree that it is good 
practice to develop an extensive poo! of good test 
items that can be used again. This dpes-not mean 
that Old tests should be reused in their entirety; 
this is bad practice and should always be avoided. 
What it doesjpean is that teachers might possibly 
take good questions from exams and keep a file of 
them for future use. One useful procedure for 
doing this is to keep a record of test questions on 
index cards. 'The- test question and possibly the 
o' jective is put on one side of the index card and 
other pertinent information such as "item 
difficulty" and "date given'' on the other side. 
These item banks can be categorized into different 
subject areas which aici the- instructor in assuring 
that his tests adequately sample the content 
covered during instruction. 



Writing Essay Tests 

Most experts on test construction shy awav from 
"\ using or suggesting the use of essay tests even 
though they seem to have distinct advantages in 
certain situations. Testing recall rather than 
^^cognition, testing higtaer-level mental processes, 
''^^^{e-Q-. reasoning, inference, problem solving) and 
testing originality or creativity are some cf the 
presumed advantages of the essay test, but critics 
of the essay test usually conclude that its bad 
points f^r outweigh its good ones. Their 
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ccmnniion js that ohinciive lests, <f properly 
constructed, can lost recnlL creanvity and higdef 
mental processos as well as essay examinations and 
"can do it much more reliably and fairly at less 
cost. 

Because the essay test does seem to have certain 
advantages and because il is often used and will 
no doubt continue to be used, some of its 
shortcomings along with some ideas on how to 
improve essay tests by min'mi/ing these probli^ms 
will be ruviewed. Some of the G^seniial 
characteristics of the essay examiiiatlDn are: 

1 . The student organizes his own answers, with 
a minimum of constraint. 

2. The student is required to produce the . 
answer rather than merely to recognizee it. 

3. The student must integrate ideas and facts 
into a coherent whole. 

4. The student uses his own words and ideas, 

5. He answers a small number of questions. 

6. Answers can.be of all degrees of correctness. 

Mn these ch"aracteristics lie both the strengths and 
weaknesses of the essay examination. 

Psiiability of Essay Tests. The number one 
pioblem of the essay test ts consistency of 
measurement (scoring) . Nurrreirtff'^s studies have 
■shown that scores or grades obtained at one point 
in time ^n essay exams seldom coincide with 
scores or^grades on the same exam scored at 
another time. In these studies the instructor 
usually grades a set of nameless exams do6 thep 
•waits a few months until he has forgotten the 
scores, then he regrades the exams. Seldom are the 
same scores obtained and usually there are gross - 
differences between the scores obtained in the iwo 
settings. The problem of reliability becomes even 
more evident when different instructors (equally 
qualified) grade the same set of exam papers. 

Another reliability problem is that essay' tests 
allow time for only a limited number of items. 



The time required to answei a single 
question makes it impossible to include 
more than live or ten questions in even a 
fairly fengthy test. This tends (o result in 
what we might call a "lumpy" sampling i)f 
what the student knows. We sink four or five 
Ijig shafts Into the mine of knowledge that 
the student possesses. II these happen to hit 
pay dirt; the student does well, but if they 
hit the gaps in his knowledge, he does 
poorly. With this small number of samples, . 
chance is likely to play a relatively large 
pait. We may gel a very unfair sample of a 
particular student's knowledge.* , . 

One final problem with the essay test is that scores 
are sometimes partly dependent upon verbal 
fluenc/ and handwriting ability. A student who 
expresses himself well and has good handwriting 
will usually get a better grade than his counterpart 
who may know the material just as well but lacks 
those qualities. 

Construction of Essay Tests. Following are some 
g^iidelines for use in constructing essay 
examinations. 

1. If possible make up the questions a few 
weeks in advance, set thi'm aside and then go 
back 'and review them before final 
preparation of the e:<am. This procedure 
forces the instructor to put so ne thought 
into his' questions, doing away with 
off-the-cuff test construction. It also allows 

. the ihstructoi' the opportunity to look at the 
questions from a different perspective. Often 
^ questions that seemed tiear at the time of 
writing, are vague and inadequate when 
reviewed.' ""^ 

2. Have other instructors or team members 
review the questions. .This, like the 
procedure above, allows for a pre-test of the 
questions. Too often the instructor has in 
mind what the question and answer should 
be, but another expert looking d\ jhe 
question may not know exactly what is 
expected, let alone the student, taking ihe^ 
exam. 



**Nunnally, J, C. Educational iWeasurement and Evaluation, New York: McGraw-Hill, 1964, 



Several questions of short length are 
preferable to a few questions of somewhai 
longer length. As was pointed out earlier, 
one of the problems oi essay tests is that ■ 
they niay not sample the student's 
knowledge representatively. More questions 
allow for the valid sampling of the student's 
• content of knowledge by covering more of 
thG course iirsa. Shorter questions also have 
the effect of causing the instructor to write 
more precise questions. 

Time is a key factor in essay test 
administration. Often essay tests are tpo 
long, sampling such a wide content urea that 
the student doesn't know what is expected 
of him. Probably the main ^eason for broad 
all-encompassing questions is that the 
instructor is "unconsciously trying to 

' Compensate for what he vaguely realities is 
inadequate coverage of the content to be 
sampled." With short, precise and well 

, worded essay tests the instructor can gauge 
the time needed to answer the questions and 
allow that adequate time be given to 
cofpplete the exam. 

Another principle related to the one above is 
the use of clear terminology. This rule seems 
obvious, but it is often one of the primary 
problems with essay examinations. The 
question itself should be written as simply as 
possible. The answer to the question might 
call for specific terminology, definitions etc.. 
but if- reading and understanding the 
question is contingent upon a high degree of 
specific knowledge/ then the examiner is 
curtailing his abiliiv to find out how well the 
student knows the material. If the purpose 
of the test is to find out if the student 
knows the meaning of specific terminology 
then it would be better to use an objective 
test rather than an essay format. If the 
written question necessarily calls for such 
terminology, then it should be explained cr 
defined in the question itself. 

# 

. In short, essay questions should be stated>in 
terms that sfl students can be reasonably 
expected to understand, so that all car 
respond fairly.to the questions. 



6. Although subji^ci to a great deal ol 
contfoversy, most test exports would agree 
that all' students should lake tiie same exam. 
Students should not be olfored -a choice of 
questions to be ariswered. An example ol 
this, often used on essay exams, is "pick 
three of the five and answei." This 
procedure reduces the ability (o cofiipare 
student responses and adds one moie source 
of variability to the subjectivity and 
inaccuracy already existing. 

7. The words "what do you think/' "in your 
opinion," "write alL...you ' know 
about . . or "Discuss" almost never 
belong in an essay question to measure 
academic achievement. It is oiten very 
difficult for the student to determine just 
what kind of a response is wanted when 
questions are phrased m this manner. These 
and similar phrases usually result m the 
student writing in some more or less aimless" 
fashion until his time is up. The only time 
the use of such questions as "What do you 
think" and "In your opinion" is justified is 

^ when you want to know individual attiludp?s. 

8. In general, start essay questions with such 
phrases as "Compare." "Contrast," "Give 

original examples of," and "Explain how or 

why." These words will help to present tasks 
requiring the student to select, organize, and 
apply his knowledge. It is usually best to 
avoid starting essay "questions with such 
words as "what," "who," and "list." These 
words are likely to present tasks requiring 
only the reproduction of information. 

Following are some su ggesti ons for evaluating 
essay tests: 

I. The instructor should prepare a tentative 
scoring key at the time of the construction 
of the test. This wil| improve the objectivity 
of the scoring so that ha will not have to rely 
on vague generalized impression while 
scoring. Preparation of a key also has the 
effect of revealing weaknesses in" the 
structure of the questions that can be 
corrected prior to administration^^ 



2. Before fiD^dl scoring of the essay exam 
begins, several of the papers should be read 
with the tentative scoririg key kept m mind. 
This acts as a check on the adequacy of the 
scoring key by determining if othei answers 
or approaches are possibly as feasible. This 
may lead to an alteration of the scoring key 
before the actual scoring of the exams. 

3. Read all the answers to one question betoie 
going on to the next. The procedure allows 
for a more objective comparison of answers 
and reduces whai is called "halo pffect/' or 
the tendency for the answers written on 
previous questions to contaminate the 
judgement of the question presently being 
scored. 

4. Papers should be graded anonymously. Not 
knowing the, name of the individual increases 
the objectivity of the grade given. One 
suggestion is that the name of the student be 
placed on the back of the exam. 

5. The instructor should periodically recheck 
papers graded earlier to ensure that 
standards have not shifted. An occasional 
check of papers assures that all students are 
being graded using the same standards. 

Summary of suggestions for essay test 
construction: 

1. Make questions up a few weeks in advance 
of the exam and review them before, final 
preparation of the test. 

1: Have other instructors and team members 
review the questions. 

3. Use «several short questions rather than a few 
long ones. • 

4: Make sure time is allowed to finish the 
exam, 

5. Use clear and simple terminology in writing 
. the questions. 



6. Don't offer a choice of questions to the 
students. All students should take the same 
exam. 

7 Avoid the words "What do you think," "In 
your opinion," "Write all you know about," 
Of "Discuss." 

8. olart essay questions with such phrases as 
"Compare," "Contrast," "Give original 
Rxamp!?'^ uf," and "Explain how or why.*" 

Summary of sijgge:lions for r)r(iriing essay tests: 

1 . Prepare a model answer in advance. 

2. Read several of the pape'3 Ijefore scoring is 
begun. 

3. Read:air the answers to one question before 
going on to the next. 

4. Grade papers anonymously. • 

5. Periodically recheck papers graded earlier to . 
ensure that standards have not shifted 
appreciably. 



Writing Objective Tests 

Test' whose answers are definite, precise and leave 
little or no room for subjective interpretation a:e 
usually branded as objective tests. These exams 
can usually be scored by clerical help or machines 
as Vv'pII as by the individual who constructed the 
exam. Objective tests include such item types as 
true-false, multiple choice, matching, and 
fill-in-the-blank. 

Experts in test construction make the claim that 
|ust' uuuijt anything can be tested objectively if 
enough time and effort are exerted in preparation 
of the items. They also profess that objective tests 
can measure anything that can be measured by 
essay or oral testing and that it can be di3ne much 
more effectively. 
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Some importnnt chafaci'iristics of objective tests 
should be noted; 

1. Reliaiility. Wlinrf?ns essay examinations are 
ufien unrelial:le due to scorer 
inconsistencies-, objective exams usually 
eliminate this source of error. 

2. Scoring Economy. One very good reason 
for using objective tests and becoming 
proficient in their construction is that as the 
number of students per classroom increases 
it becomes more and more difficult to use 
anything but objective tests. With many 
students it becomes almost impossible to 
grade essay examinations reliably or 
economically. Objective tests can be graded 
using clerical help or can be machine scored 
in a very short time. This rapid return of 
results to the student has the added 
advantage of increasing the learning of the 
students. 

3. Adequacy of Content Sampling. The 
greatest virtuo of the objective test is that it 
has the potential for adequatt.iy sampling 
the entire subject-nicitter content. With four 
or five essay questions it iS difficult to cover 
the material completely so that a student's 
score is often dependent upon whether or 
not the instructor happens to hi.t his strong 
jreas. With ein objective' test of 
60 or 70 items covering the entire content oi- 
the course, an instructor can have much 
greater assurance that he has identified " 
students along the continuum of knowledge 
of course material. 

Some general ' guidelines- of objective test 
construction will be listed, followed by specific 
suggestions for writing cifferent types of objective 
lest items. 



General Guidelines of Objective Test Construction 

1. Unless you are testing reading ability or 
vocabulary, the reading difficulty of the test 
items should be- kept low. All the students 
«\ should be able to understand the question 
' even if they don't knovv the answer. 
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2. Do not lift a statemeni veibalJm from ih« 
textbook. When a student sees verbatim " 
stoLaments on a test, he if often put on the 
defensive about test takiny and it reinforces 
his idea that a student has to Tnpnu3ii/n-the- 
text in order lo be a succnssiut. student. 

,3, Stait the test with a few easy 
questions — taking ari exam can ,be a 
harrowing experience for some students A 
■ couple of relatively easy questions ai the 
beginning of the exam often have the effect 
of easing a student's test anxiety, thereby 
enabling him lo do his best on the exam. 

4. If an item is based on opinion or aiithority, 
indicate wiiose opinion or what authority 
Students, especially the more able ones, 
often know of conflicting opinions on 
different concepts and problems. Pairing ttie 
name vuiih the question eliminrites this 
potential confusion. 

5. In planning a set of items for, a test, take 
care that one item does not provide cues lo 
the answer of another item or items. 

6. Avoid the 'jse of interlocking or 
interdependent' items. These are questions 
where answering or understanding one item 
is contingent upon correctly answering or 
understanding another item. 

7. Positioning of correct responses or 
alternatives for objective tests should occur 

, as randomly as possible. Teachers should 
avoid following patterns of correct responses 
or usiiig one of the alternatives more than 
the other. The tendency to; "hide" the 
correct choice in the middle o( the 
alternatives is an example of this problem. 

8. Instructors shobld ^avoid trick and catch 
ciuestio'hs/^J^eseyijji^^^ tend to put 
students on the defensive about test taking-. 
They also give an edge to students who don't 

. know the material and so just randomly 
select an alternative as being correct while 
the prepared student makes the wrong 
response. . 
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9. Avoid ambiguity in questions. This rule 
sounds analagous to a politician saying he is 
for peace, economic stability, justice for all, 
and mom's apple pie. It is an objective just 
about as hard to achieve. One way, discussed 
earlier, of assuring clarity of questions is to 
have other instructors review the test and 
make comments. 

10. Beware of items dealing with trivia. This is a 
common fault with most classroom 
achievement tests. The primary reason for 
this is that pedantic items are much easier to 
write than problem solving, reasoning and 
inference questions, which are difficult and 
time consuming to construct. The writing of 
good, valid questions can be greatly 
facilitated by keeping the course objectives 
firmly in mind. 



True-False Items / 

True-false items are one of the more familiar types 
of objective test items. They have many inherent 
problems, a few of which are listed below. 

1. There is often a tendency for the instructor 
to take verbatim statements from a 
textbook, thus encouraging the student to 
study by rote memorization of the material. 

2. Students often feel that true-false items do 
not test what they actually know or can do. 

3. True-false items often picture the world as 
black or white and not as possibilities on a 
continuum. . 

4. Because there are only two alternatives to 
each-question some students do n^uch better 
than others simply because chance plays a 
proportionately larger rote. 

5. It takes significantly more questions to make 
a true-false test as' reliable as a 
multiple-choice exam. 

6. The biggest objection to true-false items is 
that they are extremely difficult to write. As 



often as not, subtle hints within the question 
reveal the answer without the student having 
to know anything about the concept being 
tested. 

Following are sdme cautions to be observed in 
writing true-false items: 

1. Attempt to make each item unequivocally 
true or false. Often the students who know 
the most about the subject area have the 
hardest time deciding if the item is always 
true. 

2. Beware of words or phrases that give cues to 
the probable answer. Statements that 
contain "all," "always," "no," and "never" 
are so all-inclusive and represent such broad 
generalizations that they are likely to be 
false. Statements involving such items as 
"usually" or "sometimes" are usually true. 
These cues are referred to as "specific 
determiners" and should be avoided in test 
preparation. 



Examples 

T F . The choice of the route of administration is 
dependent only on the ease of 
^accomplishment. 

IV Biologies are always dosed per pound~^of 
body weight. 

rF Oral administration is usually the safest 
route. 

3, Beware of ambiguous and indefinite terms 
such as "frequently," "greatly," "to a 
considerable degree/' and "in most cases." 
These terms mean different things to 
different people. ^ 

4. Negative statenrients should b^ underlined. 
Students hurriedly taking an exam often 
miss negative statements. 
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I ' ^ J Example 

T F Pulse rate in the dog is not as important as in 
man. , 



5. Beware of giving cues to the correct answer 
by the length of the item. True statements 
are generally longer than false ones. 

In summary, good true-false test items are 
extremely difficult to write, less reliable than 
other approaches, and often do not measure what 
they are intended to measure. Instructors should 
avoid using true-false items whenever possible. 

Short-Answer and Completion Items 

Following are some short maxims concerning 
completion items, followed by an example of a 
good completion test item. 

1. Beware of indefinite or "open" completion 
items where any number of words or phrases 
may be reasonable and factually correct. 

2. Omit only the keywords. 

3. Leave js few blanks in the statement as 
possible. 

4. If at all possible put the blank or blanks at 
- • [he end of the statement'. 

5. When the problem requires a numeiical 
answer, designate the appropriate units to be 
expressed.^ 



Example 

f.... — ^ 
Questions 5-9 ask' you to choose among five 
species of animals. 

Answer as follows: 

(a) dog/- 

(b) cat • ... . 

(c) horse . " 

(d) cow 

(e) sheep . 



ERLC 



5. The highest erythrocyte count is normally 
seen in the - . 

6. The most rapid sedimentation rate is 
normally seen in the 



7. Myelogenous leukemia is seen most often in 
the . 

8. Reticuloendotheliosis is described as an 
important disease entity in the 



9. Of the diseases of thp leukosis complex, the 
lymphogenous form is seen in over 
99 percent of affected animals in the 



This question has the advantage that it can be 
machine scored. Most often, however, teachers use 
these items so that the student is forced to recall 
the answer. If this is the case then the list of 
possible alternatives would be eliminated. . 



Matching Items 

Matching questions are much like multiple-choice 
questions except that instead of a problem 
followed by four or five possible responses, 
matching questions have several problems with 
several possible answers taken from one list. 
Following are some maxims on writing effective 
matching items: 

1. Items in a set should be homogeneous. J_hatl. ., . 
is to say that all of the items should bear 
some relationship. to each other. If they do 

not then it usually becomes relatively easy 
for the student to eliminate many of the 
alternative responses. An exaggerated 
example of this would be to incorporate 
biometric and anatomical items in the same • 
matching question.- Responses such as 
chi-square, t, F,etc. could quickly be 
eliminated from an anatomical phrase stem. 

2. The number of answer choices should be 
greater than the^ number of problems 
presented. This increases the probability that 



you wil' be assessing the student's 
knovyledge and not his ability to use the 
process of elimination bf alternatives. If for 
example a set of matching items consisted of 
four problems and four answer choices and a 
student only knew two of the four answers 
he would, still, have a fifty-fifty chance of 
getting the entire set correct. An exception 
to this is when the answer choices can be 
used more than once. 



• Example 

; Match the following gestation periods on the right 
with the animal on the left. 

Cow : " A. 86 

B. 113 

Horse " C. 146 

D, 187 

Pig E. 224 

F. 280 

Sheep G. 335 

3. The number of items in a set should . Kept 
relatively short. This facilitates keeping the 
items homogeneous and makes it easier for 
the student to find and record his answers". 

4. Response options should be listed in some. 
- kind of logical order. An example would be 

putting names- in 'alphabetical order or 
quantities in ascending or descending order. 

5 The direchons for matching cjuestions 
should be clear and u^dication should "be 
made as to whether an answer choice may be 
used more than once. 



Multiple-Choice Items 

These types of iterris are used more in classroom 
and professional testing than any other objective 
type approach. If well constructed they can be 
used to test most problems including those tliat 
require a great deal of insight, understanding, 
creativity, and thought as well as simple recall. 
Because theii flexibility is greater than other 
objective approaches they are preferred in most 
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situations. They also have the added advantage of 
being more reliable than many other objective 
measures because they provide several possible 
options or alternatives. 

Following are some suggestions for writing items 
and their accompanying alternatives. 



• The Item as a Whole 

1. The item should contain a central theme or 
.thought; If the student has to read the 
answers in order to know what the question 
is, then the question probably lacks a central 
theme. 

2. The item sliould be expressed in clear and 
precise language. Here again, this is probably 
easier said than done but clarity can be 
facilitated by having ^ others review the 
questions or reviewing your own questions 
after a suitable time lag from the time of 
original test construction. 

3. The question should be as short as possible 
and still give all the needed information. 
Teachers often feel compelled to tell the 
student somt-thing in the question even if it 
has no bearing on answering the question. 
This only takes more time and confuses the 
student. 

4. Whenever possible, questions should be 
stated in positive form. Negative questions 
have three common faults. Dne, they are 
usually easier because the resultant answer is 

, / often obvious; two, they are disturbing to 
the student who prefers to indicate best 
answers rathtr than poor or unacceptable 
solutions; and three, they are often 
overlooked when students are hurriedly 
taking an exam. 

5. The stem of a multiple-choice item should 
clearly formulate a problem. All of the 
alternative responses should be possible 
answers to a single problem. • 

6. As much of the item as possible should be 
'included in the sterri. This enhances the 
clarity and readability of the problem. 



7. Beware of double negative statemonts. They 
are difficult to read and confusing to the 
students. Following is an example of a 
double n egative question wh ich was 
confusing to many students. The correct 
response is "b" but roughly half of the 
students marked "e"_as the correct response 
because it was a correct statement but they 
had forgotten or became confused on the 
introductory statement which said "In the 
following statement which of the 5 choices 
is incorrect." In brief, half the students 
knew the answer but missed the question 
because of poor wording. 

Example 

(In thfi following statement which of the 5 choics^ 
is incorrect?) / ^ 

Atelectosis may be present a result of 
a. Insufficient production and/or supply 

pulmonary alveolar surfactant 
*b. Filling of the alveoli with exudate 

c. Compression of the lung due to 
pneumothorax 

d. Occlusion of a bronchus supplying the 
atelectatic portion of lung 

e. One of the above statements is incorrect 



1 . All the ali&rnatives should be plausible from 
UiD Statement of the problerri. Not only 
should the incorrect alternative(s) be^ 
roc.3onable but care should be taken , to 
assure they also follow grammatically from 

, the question. If the stem and the keyed 
answer sound alike vvhile the alternatives do 
nof, then the student may answer the 
* question correctly without actually knowing 
, the answer. 

2; The problem of "specific determiners" or 
cue words was discussed earlier In the 
section on • true-false items. Specific 
determiners should also be avoided in the 
. alternative responses of ^ multiple-choice 

. questions. The use of "On/y^ln alternative 
"e" is an exarjiple of a specific determiner in 
a multiple-choice item; 



Example 

The soda lime (CO2 absorber) part of the 

anesthetic should be checked? 

*a. After each surgical procedure. 

b. Before each surgical procedure. 

c. Once a week 

d. Once a day 

e. 0/7/y after several operations 



3. The alternative responses should be of 
approximately equal length. Lengthy, 
qualified responses are often the correct 
response. 

4. Caution should be used when using "None 
of the above" or "All of the above" as 
alternatives. Too often instructors use these 
as alternatives to a question rather than 
giving a little thought to additijnal good 
alternatives. "None of the above" should 
never be used when the question is intended 
to differentiate between alternatives of 
various degrees of goodness. It should only 
be used when the different alternatives are 
unambiguously correct or incorrect. Also, 
"None of the above" like negative items has 
the effect of forcing students to select an 
answer which is not an answer, 

^. The use of "all of the above" should be 
, avoided. If for example a five-choice item 
has "all of the above" as the fifth alternative 
and a student realized that two of the 
alternatives were correct then he would 
. automatically know that "all of the above" 
was the correct response. 



Example 

IBR Virus causes the following: 

a. Conjiintivitis 

b. Infections postular vulvovaginitis 

c. Abortion t 

d. Tracheitis . . 
*e. All of the above 
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Summary of Rules for objective test construction: 



General Rules 

1. The reading difficulty of the test items 
should usually be kept low. 

2. Do not lift statements verbatim from the 
textbook. 

3. Start the test with a few easy questions. 

4. Items based on opinion or authority should 
be accompanied by an indication of 

.. "whose" opinion or "what" authority. 

5. Be careful that one item does not provide 
cues to the answei of other items. 

3. Avoid the use of interlocking or 
interdependent items. 

7. Alternative responses should occur as 
randomly as possible, 

8. Avoid tricky and catch questions. 

9. Itemsl should be as clear and concise as 
possible. 

10. Avoid items dealing with trivia. 

True-False Items 

1. Items should be unequivocally true or false. 

2: Avoid specific determiners. 

3. Avoid terms such as "frequently/' "greatly," 
"to a considerable degree," and "in most 
cases." 

'4. Underline negative statements, 
b. Keep items about the same length. 



Short Answer and Completion Items 

1. Beware of indefinite or open completion 
items where any number of words or phrases 
may be reasonable and factually correct, 

2. Omit only key words. 

3. Leave as few blanks in the statement as 
possible. 

4. If at all possible put the blank or blanks at 
the end of the statement. 

5. When the problem requires a numerical 
answer, designate the appropriate units to be 
expressed. 



Matching Items 

1. Items in a set should be homogeneous. 

2. The number of answer choices should be 
greater than the number of problems 
presented. 

3. A set of matching items should be kept 
relatively short.- 

4. Response options should be listed in some 
kind of logical order. 

'5. Directions should^ be clear and specify 
whether or not an answer choice may be 
used more than once. 



Multiple-Choice Items 

1. The item should contain a central theme or . 
thought. 

2^ The item should be expressed in as clear and 
precise language as possible. 

3. The question should be as short as possible 
and still give all the needet^ information. 



4. Questions should be stated in positive form. 

5. The^stem of a multiple-choice item should 
clearly formulate a problem. 

6. As much of the item as possible should be 
included in the stem. 

7. Avoid using double negatives. 

8. All of the alternatives should be plausible 
from the statemer^t of the problem. 

9. Avoid specific determiners. 

10. The alternative responses should be of 
approximately equal length. 



11. Caution should be used when using "none of 
the above" or "all of the above" as 
alternatives. 



SUMMARY 

This paper is meant to be an aid in the 
construction of an interpretation of valid and 
reliable classroom achievement ...st .and is not 
intended to be the final word in classroom testing. 
Participants may validly differ with some of the 
precepts put forth in this paper. Good test 
construction is as much an art as a science and 
improves only with much practice, critical review 
and hard work. Each instructor must ddVelop and 
use those testing techniques that workcbjjst in his 
particular situation. 
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PROFESSION \L ASSESSMENT: AFFECTIVE DOMAIN 



H, Gardner, PIkD, 

Director, Uiidergradiiale Studies 
Department of Education^ Purdue University, Lafayette, Indiana 



It is an honor for me to participate in this 
sympsoium. This is not my first acquaintance with 
educational conferences for veterinary medicine, 
as I participated in the Purdue Workshop. Your 
profession is to be complimented. This gathering is 
excellent. You have a combination of workshop, 
symposium, and conference, h would like to 
commend the participants foQheir-Xontribution in 
helping to fulfill, the workshop portion of the 
pro- am. I think you have been working very hard 
and your results are showing. You have learned a 
great deal. 

First, I would like to specify four behavioral 
objectives. The focus of this symposium is on the 
development of mini-courses. The first 
responsibility that any program participant has is 
to specify outcomes that, as Jr. Mager says, will 
be so explicit that we will recognize them when we 
see them. My first behavioral objective is within 
two weeks, 50 percent of the symposium 
participants will have read the two handouts 
prepared by Dr. Franz Frederick and myself that 
have been distributed to you. "Second is that 
within one year, 10 percent of the symposium 
participants will have specified at least three 
behavioral objectives in the Affective Domain and 
wiirhave designed or selected at 'least one 
instrument for evaluating the effectiveness of the 
desired outcome. The th'rd behavioral objective js 
within two years, a' Symposium on Veterinary 
Medical Education will be held within the United 
States focusing on developments in the Affective 
Domain. My fourth behavioral objective states that 
within six months, 10 percent of the symposium 
participants will have selected or created an 
academic game^to include as? a part of their 
instructional process. 



It is especially gratifying to me to be discussing the 
Affective Domain with you today because five 
years ago neither Dr. Block nor I would have been 
invited to discuss the Affective Domain in such a" 
symposium. It was strictly taboo, even in teacher 
education or psychological conferences. People 
have a hang-up about designing attitudes and 
values to which all members of a given profession 
might work toward or subscribe. With this in mind 
then, I find the attitude ai.d openness of this 
conference refreshing. I wish that we we^^ engaged 
^in such a symposium for teacher educat'on at 
Purdue. What I m really concerned with in this 
portion of the symposium is 'fuzzy assessment. 
These are fuzzies that Dr. Mager referred to. Every 
speaker has alluded to this area of development, 
but no one has defined it. 

I decided rathf^r than focusing on the evaluation of 
outcomes in the Affective Domain, that perhaps I 
ought to develop a concept of what the Affective 
Domain is, what it means, what some of the 
implications are of objectives and instruction in it. 
Our speakers have talked about the Affective 
Domain. For example. Dr. Luther Terry said that 
we must research the relationship of pets and 
psychotherapy and we should check closely to 
determine really what the significance of the role 
of companion animals is in suoporting good 
mental health, Dean Armistead di.i an excellent 
job in challenging us for the future. He made 
several statements referring to the Affective 
Domain citing the need to develop confidence in 
this area. He said veterinarians must change their 
attitude toward the importance of teaching in the 
Affective Domain and they must learn to value it 
more highly. The veterinarian must value his 
commitment to service, fiot just his commitment 

/ 
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to money, the benefits ttiat might accrue from 
being a member of the profession. Further, he 
stated that the veterinarian must change the 
consumers' attitude toward the image of the 
veterinarian in the United States, to viewing the 
veterinarian as a real friend and courcelor; 
someone w»]0 can be very supportive. He must no 
longer continue to be viewed as an impersonal 
scientific specialist. Dr. Armisiead said that the 
veterinarian must develop a desire to preserve 
animal health and must nc_ longer focus on 
treatment of diseases. Further, he said we must usa 
students to assist in designing curriculum as well u*: 
research and in creating an irrproved instructional 
process. Dr. Robert Mager, said educators who are 
trying to create changes in instructional design are 
usually operating in a hostilp environment. We 
must be concerned v\ith this type of environment 
and make every effort to modify it. ^ 

I have had many conversations with a number of 
you and you are concerned about the Affective 
Domain. You are concerned with the attitude of 
your students. If you start toying around with 
mini-courses and* specifying behavioral objectives 
what will your students think about you? Will 
they appreciate what you are' trying to do or will 
they sort of Jismiss or laugh at you? How about 
your colleagues, will they support what you are 
doing? Will they add their support? How about 
your dean or department head? And how about 
thd University? If we are operating in a hostile 
environment (and most oif us are employed and 
provide services within a large corporation, college, 
or university), do the dean and the department 
head really have much choice in modifying the 
hostile environment or will he be nonsupportive? 
In other words, is all of this going to be worth the 
effort? You are also concerned about the 
professional attitude of future veterinarians. Can 
you really work within the Affective Domain and 
bring about changes in altitudbb and \j'ues that 
are now held by future veterinarians, so that they 
subscribe to the kind of beho^'ior that 
Dean Armistead outlined earlier? 

What do we mean by the Affective Domain? I 
would like to refer vou. to the handbook entitled. 
Taxonomy of Educational Objectives by 
David Krathwohl, Benjamin Bloom and others. 



Within the concept developed by Krathwohl and 
Bloom of the Affective Domain, affect includes 
appreciations, attitudes, values, interests, and 
adjustments. It focuses on the process of 
internalization. That is, and I quote, "the inner 
growth that occurs as the individual becomes 
aware of a. .J then adopts the attitudes, principles 
codes, and sanction: that become a part of him in 
forming value judgements and guiding his 
conduct." I have provided you with a handout 
that explicates the levels of the Affective Domain. 
This is taken from the handbook of educational 
objectives in the Affective Domain. Vou will 
notice that these levels are numbered and beoin 
with receivi. 3 and then move up through 
responding, valuing, organization, to the fifth level 

. which is characterized by a value complex. When I 
refer to the process of inernalization, I mean that 
as you more fully internalize the behavior that ycj 

^ observe and subscribe to, then you move up to a 
higher level of the Affective Domain. In other 
words, as inner growth occurs you more 
completely adopt certain values^^^^attitudes, 
appreciations. You can look at this when vou have 
a. bit more time and I think that it perhaps will 
help clarify" part of the concept of the Affective 
Domain. 

Now of course there are some obvious problems in. 
dealing with measurement , in the -Affective 
Domain. Currently, at least, we cannot directly 
measure affect. Affect lies within the individual 
and may not be directly observed. We may 
attempt to measure some characteristics or 
properties of affect, however, this has pitfalls too. 
In reality we measure traces of behaviors which 
occur as a result of these characteristics. I would 
like for you to refer to the first overlay that I have 
made. Here you will notice at the bottom of the 
transparency traces of behaviors. Moving up, you 
will notice characteristics for properties of those 
behaviors and at the top you will notice affect. My 
example suggests that one trait of behavior might 
be someone- who is always striking someone, 
hitting them; pushing them, shoving them, and we 
characterize this by the term hostility. This at the 
affect level means that theie is some feeling 
toward a person, object, or c ent, In this case from 
my point of view, it is a negative feeling. 
Interestingly enough for you and me at the 
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professional level, there are people whom we 
would class'fy. or characterize as hostile who strike 
others verbally, not physically, and they getaway 
with it every day. They really are manifesting 
some feelings toward a person or persons or some 
object. 

What we are concerned with is the measure of 
eight areas of affect and I iiave outlined these on 
\he second overlay. V/t zie emotionol set, anxiety 
level, motivation level, attitudes, and attitude 
Clusters, values, organization of values, problem 
analysis on the basis of both cognitive and 
affective concerns, and personal value systems. 
Now the question arises, what can yr^u do as i 
veterinary medical educator? First of all, what can 
you du in regard to student response to your class 
or a new method of instruction. First, I would 
suggest that you read Dr. Robert Mager's 
publication. Development of Attitude Toward 
Learning. Within this he talk^ about, in general, 
two types of responses within the Affective 
Domain, approach and avoidance responses. Now 
for example, my young son responds very ^arly 
and openly. I think you can detect hisattituae. If 
I say, "Come on son, I wnrit you to eat spinach 
and green beans," he usually pushes his chair back 
and says "yuk." But if I say come, finish your 
meal and then we will have a nice chocolate ice 
cream sundae with marshmallow and a marchino 
cherry, he leans forward and says, "Dh Boy, 0. K., 
Yum, yum." In the first case, of course, we have 
an avoidance response and the second one, an 
app''oach. Now what you should do is to specify 
an objective that clearly defines trie increased 
number of approach responses or decreases the 
number* avoidance responses over a given period of 
lime. Then tabulate these. In other words make a 
list of responses within your classroom .toward 
your teaching or a new approach and also 
avoidance responses. Then as you utilize a new 
approach, note these and tabulate Ihem. Talk w»th 
them. Make some modifications and try to 
increase the number of approach ^soonses and 
reduce the number of avoidance responses. This is 
one way of measuring outcome in ihe Affective 
Domain. 

If you really want to improve in the number of 
approach responses, I think you can use 



Dr. PoF'lethwait's techniquo Hn is relaxod ufid 
open. He demonsiraled a sincere r^'spocl for each 
person's opinion and increased the rmmbtir of 
approach responses toward hmi dirint^ his 
presentation to you earlier. Is tlicrn iuiyone here 
who really felt reluctant to approach lum 
following his session during the break? I do not 
think any of you could answer that you really had 
any feeling of reluctance. You were willing to 
approach him. Thus, iry a new method or a new 
techr que, and then tabulate the number of 
approach and ih'^. number of avoidance responses. 

A second thing thai you are concerned with is how 
you can jevelop and improve the professional 
attitudn of the future veterinarian, toward his 
clients, "irst, what must be defined is "What is a 
good professional attitude?" What does a 
veterinarian do in workirig with a client? Does he 
smile while giving a dog a shot? Does the dog 
smile? How does he treat the client? Some people 
have alleged that medical knowledge is not a 
measure of success, rather a pleasing personality. Is 
this real.'y a necessary quality? How important is 
it? Do you do anythinq to prepare a veterinarian 
with a pleasing personality? In fact, should you? 
How should the future veterinarian behave toward 
his colleagues? Should he accept referral cases with 
ease and without making comment? Do you ever 
comment negatively when you are working in the 
clinic in front of aspiring veterinairans? When an 
old cow is- referred to you from a veterinarian 
within the community, and this old gal is jus' 
loaded with burrs and manure, do you accept it 
profpss'onally and ot the basis of the report that 
the veterinarian has submitted? Do you encourage 
these young veterinarians to cpenly * e!< help from 
their colleagues if it is a tough case? Do you feel 
free to consult other faculty when you are in the 
clinic or do you view it as failure on your part to 
have to consult with another member of your 
facult,? And if you are consul '^^d, how do you 
respond? Do you look at your colleagues in 
amazement? Do you look at him negatively? Do 
you find this an approach or an avoidance 
. response? 

Remember that when you are in the business of 
changing attitudes or shaping attitudes or values 
that one of the most affective means of changing 



•95 



attitudes and valuns over a period of time is by 
becoming a mode) for d young person. If someone 
identifies with vou and the profession through 
you, then he will begin to manifest your behavior. 
If a lady brings in a dog with a bad back and states 
she has been to three veterinarians and they have 
•not been of any help, do you tell her you will cure 
her dog, that you have a new method? Do you 
suggest or tell her that you are a chiropractic 
adjustment specialist on spinal columns of-dogs? 
Dr do you suggest that you are really not surprised 
considering the inability of the other veterinarians 
in the area? Do you tell a client or an individuaT 
that you have Dr.M's secret serum for irealirig 
mastitis? Attitudes and values develop over a long 
period of time and you must serve as good models 
if you expect the candidates for the profession to 
assume the same behavior. 

How do you assess attitudes? Specify behavioral 
objectives. Be precise and describe them and then 
design an attitude inventory. I would suggest the 
scale as described in the paper that Dr. Frederick 
and I developed under Dr. Frederick's name, 
"Measurement in thn Affective Domain." Ask the 
faculty and students to complete this at the 
beginning and at the end of each year, or each 
semester. How about attitude toward the 
profession? Should the good veterinarian belong to 
a professional organization? Should he be active? 
Should he be inactive? Should he. not be 
concerned? In teacher education we have an 
organization referred to as the Student National 
Education Association. I am sure this would never 
happen to you in regard to the A.V.M.A., but we 
liave a terrible time trying to find a faculty 
member who will spend one night a month 
working with students setting up meetings so that 
they can become acquainted with one of the 
future organizations to which they will subscribe 
and pay hundreds of dollars, in a lifetime in 
pr.fessional dues. Dne of the problems that we 
ha^e is that our professors schedule hourly or 
vigorous laboratory examinations at 8:00 in the 
morning following the night of a regularly 
scheduled monthly meeting of the S.N.E.A. I am 
sure that you probably do not have this problem, 
but if you do, then how in the world do you 
expect them to develop a more desirable attitude 



toward the profession than you? How do you 
assess this? 

First of all, again, specify some behavioral 
objectives and design attitude inventory. Have the 
students complete it and then complete it again 
after a period of time, a year, and compare, and 
you will note a trend occurring. Df course, there 
are other things ttiat happen. You will find that 
more professors will begin to give priority to 
meetings, one objective that you may specify, for 
example, is that no examination will be held the 
day following a meeting of the student chapter of 
the A.V.M.A. I think you can think of other 
objectives that will make a significant difference 
immediately. I would also suggest tliat you 
develop a pool of the behavioral objectives among 
your colleagues and with colleagues at other 
universities. There is no need really for each of 
you to reinvent the wheel every time you gain 
insight into a problem of your concern or a 
method of determining outcomes that have been 
achieved in the Affective Domain. 

Another question ttiat you have raised is, how can 
I change the attitude of my colleagues toward the 
need for improving the. instructional process? This 
is not a simple matter. You are talking abQut 
achieving something that is at the highest level" of 
the Affective Domain. What you are really 
attempting is to modify your colleague's 
philosophy of education. He has experienced 
success He has a damn good memory and nothing 
succeeds like success. If there is one thing that I 
know about you, I know that you have a 
tremendous memory. Each and every one of you. 
The rules of the game of the past have required it 
and they still do. I think I understand your 
program well enough to assume that if there is one 
■ thing you are most profident at, it is memorizing 
concepts, ideas, facts, kom^ledge. This is at the 
lowest level of that Cognitive Domain, if you 
check out handbook number one by Krathwohl 
and Bloom. 

What you are asking your colleagues to do (who 
are less motivated by the establishment of 
mini-courses and specifying objectives in ,the 
Affective Domain) is to change the rules of the 
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game, to be concerned about behavior, to be more 
concerned about the application of facts and 
knowledge and to be more concerned about the 
attitudes of the professional, or just as concerned 
as we are about cognitive development. So you are 
asking them.^10 really reorganize their whole value 
complex that they had internalized. 

How do we assess this? I think that deans and 
department heads have a responsibility to devise 
anj attitude sca^e or use- one that has been 
developed. Any one of the three described in the 
one paper may be of assistance. They should work 
10 gather evidence to show growth and specify 
some behavioral objectives which in the very 
process will result in a great manifestation of 
support of the dynamic and creative menibers of 
your staff.'' 

In the short time I have spent with you this 
ahernoon what I have tried to do is, to fulfill these 
behavioral, objectives. I have tried to summarize 
some of -ihe statements and comments that 
demand attention in the Affective Domain. 1 have 
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given you one brief and concise definition of 
Affective Domain to stimulate your thinking. I 
have tried to outline some of the problems that 
you will encounter and have encountered in 
-specifying as well as assessing behavioral objectives 
in this domain I pointed out the need for 
assessment of development in the Affective 
Domain and sorne suggestions for achieving the 
assessment. I have provided you with three 
handouts. They should assist you in developing a 
clear concept of the Affective Domain and 
eliminating some of the fuzz on the fuzzy. One of 
the handouts provides you with instructions for 
completing an academic game. Be sure that you 
read that before the 8:15 a.m. session tomorrow. 
We will play it then. Hopefully, as you design 
mini-courses in veterinary medicine, you will 
become concerned with development in the 
Affective Domain, the process of 
internalization: "The inner growth that occurs as 
the individual becomes aware of and then adopts 
the attitudes, principles, codes, and sanctions that 
become a part of him in forming value judgements, 
and in guiding his conduct/' 
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Although the concept is old, one of the most 
powerful forces shaping present educational 
theory and practice is Bloom's (1968) notion of 
mastery learning. Mastery learning proposes that 
all or almost all students can master most of what 
we . teach and, challenges us to find classroom 
strategies whereby this proposition can be realized. 
The basic problem in the design of these strategies, 
though, is how to define mastery. 

Yesterday Dr.Mager dealt with the first aspect of 
this problem, namely, how to define what students 
are expected to master in terms of a set of 
instructional objectives. And earlier today 
Drs. Hedges, Smith and Gardner dealt with the 
second aspect, namely, how to translate these 
objectives into a set of evaluation experiences 
(e.g., test items). In this session, therefore, I wish 
to grapple with the last, but most overlooked, 
aspect" of the problem, namely, how to set the 
mastery standards against which a student's 
performance can be judged. 

My presentation will consist of three major parts. 
First, one reason why we need the best possible 
mastery standards will be suggested and then some 
characteristics of an ideal standard will be 
proposed. Next, some practical techniques for 
setting standards which begin to approach this 
ideal will be developed. Finally, a few of the many 
issues you are likely to encounter as you wrestle 
with the standard-setting problem will be 
sketched. 



The Need fur Good Standards 

Let me begin, then, by turning to the question, 
"Why do we need the best possible mastery 
standards?" The answer is simple, Withoat such 
standards it is impossible to elicit optimal learning* 
from each student. Regardless of the quality of 
" "one's evaluation instruments, poor standards will 
cause some, perhaps many students' learning to be 
misevaluated. These misevaluations make it 
impossible to design either group-based or 
individualized instruction of optimal quality for 
each learner (Block, 1972; Glaser and 
Nitko, 1970). Further, in those cases where the 
misevaluations underestimate a student's learning 
capacity, they help to destroy his desire to learn 
and even undermine his actual learning capacity 
<Block,1972). 



The idea Standard 

Having suggested that good standards are a 
necessary condition for eliciting optimal student 
learning, let me now propose some characteristics 
of an ideal mastery -performance standard. First, 
the standard must derive from the same values that 
guided your choice of instructional objectives. Too 
often we forget that the setting of performance 
standards, like the choice of objectives and testing 
instruments, depends on values (Stake, 1970). 
Hence, by encouraging students to achieve specific 
objectives to particular standards, we are in 
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essence socialising them to the values inherent in 
the standards and to those inherent in our 
nbiectives. Unless these two sets of values are 
congruent, then students will be socialized to 
inconsistent values. Unfortunately, in resolving 
these inconsistencies, many students will accept 
the. values implicit in our standards rather than 
those explicit in our objectives. For example, 
teachers often select objectives stressing the value 
of cooperation in learning yet use standards 
emphasizing the value of competition. 
Consequently, their students are sociali;?ed to both 
values. Many of these students often reso've-this 
conflict by accepting the notion that learning is 
basically competitive. . . 

The second characteristic of the ideal standard is 
that it must be non-arbitrary in thQ sense that its 
superiority vis-^-vis other possible standardscan .be 
logically and empirically defended 
(Bormuth. 1971). Tnlhe past we have tended to 
choose our mastery standards arbitrarily as if one 
were just about as good as another. Many of us. 
for example, have used a score of 90 percent 
correct s a standard of A work when a scorn of 
85 or 95 percent might have served just as well. 
Similarly.' many of us have used a convenient 
break in a score distribution to distinguish 
A from B work, but we have used different 
breaking points from year to year. 

This practice can no longer be justified. Recent 
research reveals that some standards are distinctly 
better indices of the adequacy of a student's 
learning than others and that the attainment of 
different standards can have very different effects 
on different dimensions of a student's learning. 
For example, two years ago I taught eighth graders 
a three-unit, matrix arithmetic sequence 
(Block. 1972). One group of students learned 
under no requirement that they attain any 
particular per unit performance Jeyel. Four other 
groups learned urider a per unit performance level 
requirement, but each group learned to a different 
level, i.e.. learned either 65. 75. 85. or 95 percent 
of the4jnit's material. The findings indicated that 
only, attainment of the 85 and 95 percent levels 
did have significant positive effects on student 
learning. But attainment of the. 85 percent level 
yielded maximal "learning when learning was 
characterized by affective learning criteria and 



less-than-maximal learning when learning was 
characterized by cognitive learning criteria. The 
reverse was true for attainment of the 95 percent 
level. The implication of findings such as these is 
clear. We simply cannot afford tc^centinue 
requiring students to reach performance standards 
whose attainment has little effect on various 
dimensions of their subsequent learning when 
standards can be set whose attainment will fiave 
maximal effects. 

The third characteristic of-the ideal standard is 
^that it must be absolute. By "absolute"-J mean 
three things. First, the standard must separate, 
students into two groups only — "true" masters 
and -"true" non-masters (Harris. 1972). This 
implies that the standard selected must be set in 
such a way that the number of students 
misclassified is zero os as near zero as possible. 
Second, the standard must be calibrated solely in 
terms of a student's learning rather than in terms 
of his learning relative to his peers. Finally, the 
standard must serve as the only hardstick for 
determining the adequacy-inadequacy of each 
student's learning. 

**• 

The fourth characteristic of thb ideal standard is 
that it must be used only in conjunction with 
performance data provided by so-called 
"criterion-referenced" (Glaser and Nitko. 1971) 
testing instruments. This characteristic ensures 
that whether the student is or is not judged to 
have mastered a given segment of instruction 
depends on what he has learned rather than /joiv 
well he has learned relative to his peers 
(Block. 1971). It also ensures that attainment of 
the standard has a clear meaning in terms of a 
well-defined group -of objectives (Glaser and 
-Nitko. 1971). v: 

The fifth and final characteristic of the ideal, 
standard is that it must be feasible; that is. the " 
standard should be attainable, given reasonable 
investments of human and non-human resources 
and teacher and studentjime. 



Some Possible Techniques 

We are now in a position to consider some 
techniques for setting standards which begin to 
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approach this ideal. In recent years a number of 
promising empirical techniques have evolved. But 
because these techniques require some background 
in statistical'and psychometric theory, I wish to ' 
skip them here and to focus instead on some less 
technical approaches. 1 

Summative Standard Setting Techniques 

The first group of techniques is designed to set 
-what might be termed "summative" mastery 
performance standards. These are standards used 
to judge the adequacy-inadequacy of each 
student's learning on the basis of his performance 
on a summative evaluation instrument (Bloom, 
Hastings, and Madaus, 1971) administered at the 
end of a course, semester, or term. 

Expert Judgment. One technique utilizes experts' 
judgment and is patterned after an approach used 
by the Office of Strategic Services (1948) in World 
War, 11 to determine individuals' fitness for 
high-risk espionage missions. First, a panel of 
experts, the standard setting team, would be 
chosen. Then, each expert would be asked to 
. arrive at a reasonable standard and to indicate the 
reasons for his choice. Finally, the experts would 
be asked to arrive, as a team, at a consensus 
standard"""via~ the process of deliberation 
(Walker, 1970). 

Though similar to discussion, deliberation has a 
practical (Schwab, 1969) goal — in our case, the 
selection .of an appropriate standard. The 
deliberative process would take place in three 
phases. First, each expert would -propose his 
standard and, more importantly, the reasons for 
his selection. Second, the experts would argue 
about the various alternative standards and 
especially the reasons which underlie ti^ese 
standards. The purpose of this phase would be to 
extract the basic criteria the experts used in 
arriving at their standards and to submit each 
criterion to public scrutiny. Finally, the team 
would be asked to arrive at a set of defensible 
criteria from which the standard should derive and 
. ' to select that standard which best meets these 
criteria. 



The quality of the standards yielded by this 
technique obviously depends on the composition 
of the standard setting team. The more the team is 
representative of those diverse qroups who have 
major .stakes in the quality of students produced 
by our veterinary colleges, the better the standards 
are likely to be. For example, a good team' might 
consist not only of practicing veterinarians, health 
officials and faculty members, but also of students 
who have already covered the material over which 
the standard is to be set. FurUijx., these individuals 
might be drawn from local communities or rural 
areas and/or from the state or nation as a whole 
depending upon the locale or locales in which you 
expect your graduates to practice. 

Expert Test Results, A second technique, 
resembles the first, but here the expert's 
performance on "criterion-referenced" tests is 
partially substituted for his judgment. Once again, 
a panel of experts would be chosen. Then each 
expert would be given psychometrically sound 
criterion-referenced tests over the areas for which 
standards are to be set. The expert would be 
reqU'jsted to respond to each test item in two 
ways. He would indicate whether he believes the 
item tests material which the student must master, 
and then he would attempt to answer the item. If 
we let X equal the total number of items on a 
particular test that the panel believes cover 
material the student must master and let Y equal 
the number of these X items the panel answered 
correctly, then a perfect score over either these 
X or Y items could serve as a mastery standard. 

Besides the obvious problem of selecting a 
representative panel of experts, there is one major 
drawback to this approach — -the jmount of 
testing involved. One way around this problem is 
to use item-sampling procedures ;(see Bloom, 
Hastings and Madaus, 1972). "Suppose, for 
example, that you have a 100-item test in canine 
anatomy arid wish to set a mastery standard on the 
basis of test results from ten experts. Rather than 
giving each expert the whole test, you could break 
the test into distinct subtests. Then you could 
randomly assign one subtest to each expert. The 
responses of ten experts to these 10-item subtests 
should give you roughly the same data for setting a 
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mastery standard as if you had given each expert 
..the full test. 

Case Studies of Master Veterinarians. A third 
technique, suggested by the work of Lee Schulman 
and his colleagues at the College of Medicine, 
Michigan State University, utilizes case studies of a 
sample of acknowledged "master" veterinarians. In 
this approach, each "master" veterinarian would 
be closely studied to discover the types of 
problems he typically encounters; the variety of 
skills and knowledge which he might bring to bear 
on each type of problem; the particular set of 
skills and knowledge he actually uses and why he 
chooses this set over others; and finally, the 
manner and sequence m which he employs the 
skills and knowledge selected. On the basis of this 
information for the entire sample, one would then 
extract those skills and that knowledge which 
seem to characterize the "mastei" veterinarian. 
Next, these skills and this knowledge would be 
categorized according to particular areas of study 
or courses offered by your respective institutions. 
The criterion-referenced tests for each area or 
course would be content-analyzed to identify 
items^testing the relevant skills and knowledge. A 
perfect score over these items would constitute 
mastery. 

Use of National Boards and State Practicals. A 
fourth, technique makes use of the national 
•veterinary medicine qualifying examination 
together with the various state practical 
examinations. In this technique, these tests are 
considered to provide a general working definition 
of the competencies as "master" veterinarian j 
should possess. ] 

i 

To use the technique, first the competencies tested 
by each exam would be outlined and corrslated , 
with the competencies itested by the other exams. ; 
Those competencies. which seem to cut across 
most of the tests woulcl be taken to represent the 
competencies a "master" veterinarian should 
possess. From here on, the same procedure 
discussed in the third technique would be used. 
First, these competencies would be categorized 
according to tt\e areas of study and courses offered 
by your institution. The criterion-referenced tests 
for each area and- course would be 
content-analyzed to identify items testing 



competencies the "master" veterinarian is believed 
to possess. Finally, a perfect score over these items 
would be set as the mastery standard in each area 
or course. 

Use of Pre-existing Standards. A fifth, and by far 
the poorest technique makes use of previously 
estabtished performance standards. To use this 
technique, all that one must do for each area of 
study or course is to establish the score on the 
relevant criterion-referenced testing instru ment 
above which students' learning under non-mastery 
conditions received A's. This score would be set as 
one's mastery standard. 

Formative Standard Setting Techniques 

* — ~- 

To this point, I have tocus^d on summative 
mastery standard setting techniques. But as those 
of you who have, read Bloom's (1968) "Learning 
foi mastery" paper know, a successful mastery 
learning strategy requires not only summative 
evaluation to determine how students have 
changed with respect to a set of objectives, but 
also formative evaluation to provide on-going 
feedback about how students are changing 
(Airasian, 1971), Besides summative mastery 
standards, therefore, you will also need what 
might be termed formative mastery performance 
standards to indicate the adequacy-inadequacy of 
each student's learning at various stages during a 
course, semester, or term. , ^ ' 

While some of the summative standard setting 
techniques can be adapted for use in the setting of 
formative mastery standards, there is a far better- 
approach (Block, 1972). All mastery learning 
strategies presume that the attainment of 
particular performancejstandards at each stage in 
the student's instruction will ensure mastery in his 
learning by the instruction's completion. 
Consequently, an excellent way to set formative 
mastery standards would be to select that standard 
at each stage in the instruction whose attainment 
best maximized each student's 'likelihood of 
reaching a given summative mastery standard by 
the instruction's completion. Such standards nan 
be selected easily using elementary decision 'theory 
notions. 
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Some Issues 

Let me close now by broaching just a few of the 
many issues you are likely to encounter as you 
attempt to apply these techniques or develop your 
own. 

One issue is^ whether you should judge mastery on 
the basis of a single perforrriance standard or on 
the basis of multiple standards. All present 
standard-setting techniques arrive at a single, 
standard, usually a particular total test score, 
under the assumption that the competencies being 
tested are sampled from a relatively homogeneous 
population. Frequently, however, this assumption 
is unjustified because the competencies^ tested are 
sampled from a collection of distinct, 
homogeneous sub-populations. Rathei than 
treating this collection as if it were homogeneous 
and setting a single mastery standard, perhaps we 
should set a standard ^or each sub-population and 
judge the student's . ^rall mastery on'the basis of 
his performance vis-a-vis these multiple standards. 
Suppose, tor example, we. wished to evaluate a 
student's mastery over, a collection consisting of 
three independent sub-populations of 
competencies. We would set a standard for each 
'sub-population and compare the student's 
performance on items testing competencies 
sampled from these sub-populations against the 
appropriate standard. If he performed at or above, 
standard in each sub-population-^^he would be 
judged to have mastered the collection of 
competencies of interest. 

/— •■ 

A second issue is whether you should set the same 
standards for all individuals, \.B.,universa/istlc 
performance standards, or set different standards 
if o r different individuals, \.^., particularistic 
performance standards. Throughout American 
education there is a growing retreat from forms of 
schooling and evaluation that treat the student as a 
member of a. social system. One consequence of 
this trend is that teachers are facing increased 
pressure to assess the adequacy of their students' 
learning in purely particularistic terms. Before any 
of you unwittingly bow to this pressure, reflect on., 
the fact that universalistic standards are essential 



to the survival of our society. Neither 
particularistic nor universalistic standards alone are 
- sufficient if we are not to compromise the 
individual for the sake of the society or the society 
for the sake of the individual. 

The final, and undoubtedly the most 
thought-provoking and highly charged issue is 
whethRr you, should set mastery performance 
standards in the affective domain. In recent years, 
'there has been a distinct resurgence of the idea 
that schools must simultaneously foster the 
student's cognitive and affective development (see, 
for example. Brown, 1971). But while it is clear 
that affective development has once again become 
'a legitimate instructional objective, it is not clear 
whether we should evaluate student mastery in the 
affective domain. 

I would assert that we should. I am sure that many 
of you are startled by this statement and find the 
notion repugnant. At first so did I. But 'upon 

a examination of the issue, try as I might, I 
no. good theoretical "reason other than 
'isy for not evaluating mastery in the 
affective domain when we do evaluate it in tne 
cognitive. Consider, for example, the riotion of 
invasion of privacy and let me pose the following 
question: I n the last analysis, is it not just as great 
an invasion of the student's privacy to test his 
achievement or aptitudes as it is to assess his 
attitudes and values? The'answer, I think, must be 
yes unless one is willing to make the absurd 
argument that a student's achievement or- 
aptitudes are not his own. 

A Closing Remark 

Obviously, a brief presentation of this type cannot 
make you experts on the standard setting prpble!m. 
But as Bloom (1972) points out in a recent and 
powerful paper, modern education is wallowing in 
a pool of innocence brought about by our lack of 
awareness of the problems that beset us. I 
fervently hope that these remarks have 
contributed to your loss of innocence regarding 
. one of these problems. 
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• FOOTNOTES 



1 The reader interested in exploring these techniques is strongly encouraged to consult the papers by 
Block (1972), Harris (1972) and Millman (1972) listed in the bibliography 
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The Alfective Evaluation Game is designed to 
afford an instructional designer the opportunity to 
devise an affective instrnclional situation with a 
basic evaluation design. 

The behavioral objective for the game is: 

"Given an affective learning objective, a / 
specific learning environment, a 
'mini-evaluation strategy/ and a media 
selection matrix, the player must create a 
learning design which specifies a learning 
sequence, learner activity, the appropriate 
media, and appropriate evaluation 
procedures." 

The Affective Evaluation Game has three basic 
phases of play;. _ . 

I. The Design Phase 

II. The Evaluation Phase 

III. The Discussion Phase (optional) 

I. The Design Phase. The materials used in the 
design phase are: 

1. Objective cards — these cards describe an. 
objective which is affective in nature (e.g., it 
may deal with attitudes, values, 
emotions, etcl 

2. Learning environment cards — each of these 
cards specify a particular pattern in which 



learning could occur (e.g., small group, large 
group, individualized instruction, etc.). 

3. Evaluating mini-strategy cards — each ol 
these cards contains a suggestion for an 
evaluation strategy for affective behaviors. 
Actually these are only mini-strategies. They 
must be combined with testing concerns 
such as pre-testing, post-tesiing, and/or 
continuous evaluation in order to produce a 
"complete" evaluation strategy. 

4. Media matrix — a table for selection of 
media which identifies types of media across 
the top and specifies "properties" of media 
along the side. To use the matrix one would 
find the properties necessary for his learning 
design and then see which media ought to be 
used. The steps for use are: 

a. locate the desired property in the 
property column and glance across that 
row 

b. where the property is associated with 
medium, you will find a symbol or 
symbols in the cell representing the 
intersection of the property row and the 
medium column 

c. examine the legend at the bottom of the 
matrix for the meaning of the symbols 

d. decide v\^ich of thq media associated 
- . with that property you wish to use. 

5. The design sheet — the design sheet is 
designed to help you specify more precisely 
your design. It has f/Ve sections: 
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a. Objective section — you are to circle the 
properties which your objective possesses. 
You may i-ewrite the objective if you 
wish but you must maintain the spirit of 
the objective. 

b\ Learning sequence — you are to specify a 
series of learning steps which would lead 
a student to the objective. You must label 
each step with a number (preferably the 
numbers would be in sequential order). 

c. Learner activity — you are to specify 
appropriate learner activities and indicate 
by number the step in the learning 
sequence at which the activity would 
occur. 

d. Media and materials — re-examine your 
learning sequence and the learner activity, 
then scan the properties to identify the 
appropriate media. Specify media to be 
used including the appropriate learning 
step number. Also you will justify the use 
of media at that particular step. 

e Evaluation — you will specify an 
evaluation procedure on the basis of your 
mini-strategy card and the use of 
pre-testing, post-testing, and/or 
continuous evaluation during learning. 
You will then just.ify your strategy in a 
short paragraph and specify how effective 
. the strategy would be. 

The play of the game proceeds as follows: 

1. Each stack of cards is shuffled and one card 
dealt to each player face down. Each player 
should then. have one of each of the three 
kinds of cards — e.g., A1, B3, C2. 

2. The players each receive 

a. a media matrix, and 

b. a design sheet. 

3. The players are allowed fifteen minutes to 

read their ^cards (which define the 
boundaries of their individual learning 
design ^ problem) and to complete their 
design sheets. At the end of this time the 
game moves into the evaluation phase. Be 
sure to write the numbers of your cards 
at the top of the design sheet. 



II. The Evaluation Phase. The maienais usnd m 
the evaluation phase arc 

1. The pinyer 's completed dnstgn f()rn» and his 
set of cards 

2. The Media Matrix and 

3. The Affective Learning Design Scoring 
Sheet. 

The evaluation pliase begins when a player 
^completes his design. The player passes his design, 
cards and matrix to the player on his left who then 
becomes an evaluator of tfie design. Each player 
becomes an evaluator during this phase and uses 
the scoring sheet to evaluate someone else's design. 

'■ •• • 

The time allotted for evaluation is 10 minutes. At 
the end of this time the sets of cards; design sheet 
and matrix are passed to the next person on the 
left. This person now evaluates the materials. In a 
three player game each design is evaluated twice. 
In a four player game each design would be 
evaluated J three times. The completed evaluation 
sheets and the materials are returned to the 
designer for re-consideration. At this point the 
discussion phase may becin. 

III. The Discussion Phase (Optional). The 

purpose of this phase is to query their evaluators 
at length about the evaluations. As a result scoring 
could be adjusted or the designer could be allowed 
additional time to re-design his learning design. 

/ ; 

The game is obviously open-ended and was so 
designed to allow it to be used in numerous ways. 

IV. Example. Assume that a player received the 
following three cards: 



Objective Card: \ ^ 
Objective — 

Given a desired goal by the student and two 
or more sets of conflicting information 
regarding the goal, the student will 
voluntarily employ some version of the 
scientific method to test the information. 

A3 
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Learning Environment Card: 

Learning Environment — 

' ' You may clioose lo use either 

a. individualized instruction 

b. large group instruction 

c. independent study 

d. small group instruction based upon 

(1) ability grouping or 

(2) interest grouping 

or some combination of the above in 
order to accomplish your goal. 

Bl 



Evaluation Mini-Strategy Card: 
Mini-Strategy — 

You will be concerned nol.^ only with 
identifying an observable behavior which is 



related to the students* internal affective 
behavior (e.g.. attitudes, values, 
emotions, etc.) but also with totlying its 
occurrence. 
You might - 

1, specify that the behavior appear in 
response to the stimulus two 
consecutive times 

2. specify that the behavior occur a 
minimum number of times 

C2 



Using the cards, your task would be to specify a 
sequence of learning leading to the goal, learner 
activities, media and materials that would be 
utilized, and an evaluation design to determine the 
effectiveness of learning. You may use the material 
from either of the two Mini-Reso»rces, "Some 
Functions of Testing" and/or "Measurement in the 
Affective Domaiii." 
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SOME FUNCTIONS OF TESTING 

H, Gardner, Ph.D. 
Director^ Undergraduate Studies 
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Department of Education^ Purdue University^ Lafayette, Indiana 



The major function of testing is to determine the 
state of something (say a student) with respect to 
some goal, objective, or value. Within this major 
function there are at least three subfunctions: 

1 . To assess the ability of a student 

2. To assess the.pe''formance level of a student, 
and/or 

' 3. To assess the effectiveness of the learning 
environment (e.g., the learning materials, the 
instruction, etc.) 

Table I shows the relationship of the techniques of 
pre-test, post-test and continuous evaluation to 
these three subfunctions. The table also indicates 
the nature of questions which may be answered by 
using these techniques. 

Now let us examine Figure 1. Figure 1 represents 
an attempt to show schematically the purposes of 
the evaluation procedures shown in Table L 

From an examination of Table I and Figure 1, one 
may observe that the following questions may be 
answered by the mini-designs of evaluation shown 
in Table I. 

1 . What are the abilities of a student? 
SeeA1.1,A1.2,A1.3,in Tablei. 



2. What does the student know before 
instruction and/or learning? 
SeeB1.1,B1.2,B1.3 in Table I. 

3. What does the student know after 
instruction and/or learning? 

See 82.1,82.2 in Table L 

4. How much does a student know as a result 
of instruction and/or learning? 

See 83.1 in Table L 

\. 

5. How much does a student know at rach 
stage in his learning? 

See B4.1 in Table I. 

6. Does the student learn as a result of the 
^ instruction and/or learning materials? 

See C 1.1 in Table I. 

7. ' How can one determine which parts of the 

learning sequence need revision or 

replacement? 

See C2.1 in Tablet. 

There bie also questions which may be asked 
about the validity of the learning and/or 
instruction, the reliability of learning and/or 
instruction, and the validity and reliability of the 
tests themselves. These questions are however 
reserved for another Mini-Resource booklet. 
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TABLK I. Mini-Designs Involving Pre-Tesls, Post-Tests and Continuous Evaluation 



Assess Effectiveness of 
Learning Environment 
(e.g., learning materials, 
instructor, etc.) based upon 
Assess Ability Assess Performance Level performance level assessment 



Pre-Test only: 

A1.1 10 determine whether to 
allow student to enter 
learning sequence 



A1.2 to determine which 
moterials or sequence 
. student should use based 
' on level of his ability (a 
"branching" type of 
decision) 

A 1.3 may be used to provide 
information as a co-variate 
for a co-variance analysis 
on order to extract the 
effect of ability. A 
learning gain as measured 
by subtracting pro-test 
scores from post-test 
scores may be the result of 
( 1 ) the instructional 
environment, (2) the 
original ability of a learner 
or (3) some combination 
of (1) and (2). 

B y using p re-test 
information on ability as a 
co-variate one may 
determine whether Jhe 
gain is due to the learning 
environment or ability or 
both. 



Pre-Test only: 

B1.1 to determine the students' 
performance with respect 
to some norm 
(e.g., standardized test) 

B1.2 to determine whether 
student already possesses 
the terminal behavior(s) 

B1.3 to diagnose students' level 
of performance in order to 
determine areas of 
weakness and strength. 
The purpose is to then 
prescribe supplementary 
learning experiences. 

Post-Test only: 

B2.1 to determine the level of 
student performance after 
completion of learning 
sequence. 

{NOTE: This does not tell 
whether the student 
learned'from the materials 
or whether the student 
now performs more poorly 
or how much he learned.) 

B2.2 to diagnose students' level 
of performance in order to 
determ ine areas of 
weakness and strength. 
The purpose is then to 
prescribe supplementary 
learning experiences (tends 
; toward rerriedial work). 



PreTest and Post-Test: 

C1.1 to determine how much a 
student learns (pre-test 
subtracted from post-test 
score). This answers only 
the question of overall 
effectiveness of the 
learning procedures. It 
does not permit one 'to 
identify the effectiveness 
of portions of the learning 
procedures, 



Continuous Evaluation: 
(involving pre-test and post-test) 
C2.1 to provide information 
about student 
perf 0 rmance as he 
proceeds through the 
learning sequence as well 
as beginning and ending 
performance. Allows one 
to identify portions of the 
learning sequence and/or 
environment which need 
revision or improvement. 
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Pre-Test and Post-Test: 

B3.1 to determine how much a 
student learned (a gain 
sc ore arrangement 
e.g., subtract the pre-test 
score from the post-test 
score). This information' 
answers the questions 
raised in Post-Test 
N0.B2J. 

Continuous Evaluation: 
(including pre-test and post-test) 
B4.1 to provide V information 
about student 
performance as he moves 
through the learning 
sequence. Allows one to 
profile a student's 
pe-formance by various 
aspects of the learning 
sequence. 



FIGURE 1. Schematic showing Relationship of Testing and Learning 
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MEASUREMENT IN THE AFFECTIVE DOMAIN 
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According, to Fred Kerlinger (6, p. 413), 
"measurement is the assignment of numerals to 
objects or events according to rules." in the 
cognitive area of learning the specification of these 
rules (e.g., hypotheses, theories, and/or 
relationships between what the learner "knows" 
and what he "does" or "can do") are relatively 
straightforward. The measurement process is the 
same with regard to affective concerns but the 
"rules" are considerably rDore difficult to specify. 



A Major Problem 

The major problem in measurement in the 
affective domain lies in the fact that currently, at 
Jeast, we cannot directly measure "affect." 
"Affect" lies within the individual and may not be 
directly observed. We may attempt to measure 
some characteristics (properties) of affect only to 
find these to be ephemeral too. In reality, we 
measure tracesjor. behavio-sj which occur as a 
result of these characteristics. Th'r chain of 
relationships is shown in schematic lorrn in 
Figure 1. 

To "measure" affect, we must (1) meas* fb some 
traces of behavior, (2) infer some characteristjc(s) 
which influence the occurrence of the behaviors 
and (3) infer a relationship of thecharacteristic<s) 
to some major area of affect. This would not be a 
particularly difficult problem if the inferential 
relationships were as straightforward asshown-in 
Figure 1. In many cases these inferential 
relationships may be as shown in Figure 2. 



Let us consider two cases based on Figure 1. First, 
suppose we choose behavior B] as evidence of 
affective concern No. 2 and proceed to measure 
B]. Now if Bi occurs, are we justified in 
concluding that the student has affective concern 
No. 2? No, because B] also is a reflection of 
cognitive concern No. 1. In this case B] may be 
due to either one or some interaction of both. 
Second, it may., very well be the case that in order 
to demonstrate the "existence" of affective 
concern No. 2 that we will need to have behaviors 
Bi and B2 and B3 occur. In this case the 
occurrence of only B2 may not be adequate 
because it may occur as a result of some irrelevant 
process or concern. 

The major problem of measurement in the 
affective domain as "Are the responses (behaviors) 
a function of the affect rather than some other 
variable or process?" 



Major Affective Concerns 

For the purposes of this paper we shall assume the 
following areas of affect: 

1. emotional set 

2. anxiety level 

3. motivation level 

4. attitudes and attitude clusters 
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FIGURE 1. Inferential Relationships of Trace Behavior to Characteristics of Affect 
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FIGURE 2. Example of the Complications of Inferential Relationships 
Between Behavior Traces and Characteristics 
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5. values 

6. organization of values 

7. problem analysis on the basis of both 
. cognitive and affective concerns 

8. personal value systems. 

The areas listed above stem from various sources 
including research and the Taxonomy of 
Educational Objectives: Handbook II: Affective 
Domain. 

Every learner has a set of emotions ^and these 
emotions have to some extent been "attached" to 
various stimuli and situations. Emotional set refers 
to the extent to which emotions are aroused by 
nsw stimuli or situations. 

It is difficult to change strong emotional sets so 
our purpose would be to help the learner to attach 
"positive" emotions to those stimuli and situations 
important to a particular field. 

Anxiety level has an important effect on all 
learning. Generally speaking, learning efficiency 
improved as anxiety level increases. There is 
however a plateau and regression or inhibition 
effect. The importance of anxiety level can be 
readily observed — e.g., if a person is at a very 
high'^anlciety' level then his learning performance 
maybe imparled. 

Motivation has been presumed to be more critical 
than .specific ability and indeed this explains 
overachievement. Underachievement has likewise 
been explained as the result of. ability without 
motivation. Motivation then appears to be a 
critical ingredient to any learning situation. 
Research has also demonstrated a relationship 
between motivation and anxiety. 

An. attitude is basically {he predisposition to act, 
react, think, feel or perceive in a certain way. 
Attitudes appear to be related to emotional set, 
anxiety level and motivation level. Attitudes 
represent "approaches" or "avoidances" to 
generalized sets of stimuli or. situations. It may be 
cr-itical-to learning to know what attitude a 
student has toward stimuli or situations similar to 



those 'critical to a particular field Qi_ tratnrng. 
Negative attitudes (avoidance) may cause an 
inhibiting transfer to the new stimuli or situations. 

A value is a conceptualization stemming from an 
attitude. It usually results in a generalized 
statement implying a moral or ethical 
judgement — e.g., "honesty is the best policy." A 
value is often culturally weighted. The steps 
toward the business of valuing involve (1) learning 
to accept a value say for the sake of argument, 
(2) establishing preference for one or more values 
in a field of values, and. (3) committing oneself to 
action on the basis of a value(s). These steps are 
hierarchically ordered. Training in a profession 
would be incomplete without learning to value 
based on a/l tfiree of these steps. 

The organization of values deals with the creation 
of sets of consistent values, To do this one would 
need to (1) be able to conceptualize (i.e., frame it 
as a generalized statement) a value in one's own 
terms and (2) to organize sets of values which are 
not mutually contradictory. These steps may also 
be hierarchically ordered. In order for one to be 
able to alter his own life style or professional 
direction he must be able to both conceptualize 
rand organize values. 

Problem analysis is a bjg concern in many 
professions but in the past it has;, centered on 
cognitive analysis while tending t*}- omit affect. 
Decisions may be made "rationally" and yet not 
be effective because of lack' of concern for 
affective features of the problem. ^The problem 
analysis level represents the point at which one 
begins to place different weights, on different 
outcomes because of organized sets of values. 

To be more than a "hit or miss" professional, one 
must formulate his own consistent personal value 
system (e.g., — a philosophy of life). Generally 
speaking, a person who is unable to do this will 
have proble'ms in his profession as well as his 
personal life. 

The above listed areas of affective concern are- 
arranged- in hierarchical order — i.e., each level of" 
affective performance requires some level of 
competency at each of the preceding levels of 
affect. 



Response and Measurement 

All of the levels of affect discussed previously 
relate to internal states of being on the part of an 
individual. As was mentioned previously they are 
not susceptible to direct measurement, 
consequently one must resort to indirect 
measurement. Usually the measurement involves a 
judging process either by the individual himself, by 
another "judge/' or both. In either case the learner 
is asked to respond in some fashion — usually a 
direct response or a projective response.- In the 
direct response, he is asked to judge himself and 
this tends to produce an artificial and not too 
accurate response. In the projective response, he is 
asked to judge som3one else (perhaps a 
hypothetical person or situation) and as a result 
tends to use his own emotions, attitudes and 
values for that person's emotions, attitudes and 
values. 

It has been found that when a stimulus is relatively 
unstructured and ambiguous, a learner will project 
his own emotions, anxieties, motivations, attitudes 
and values. The projective evaluation techniques 
.utilize one of the following kinds of responses. 

1. Association — the learner says the first thing 
which comes into his mind after being 
presented with a particular stimulus. The 
responses would then be analyzed to see 
what kinds of emotions, attitudes or values 
are associated with the stimuli. Another 
approach is to have the learner give as many 
associations as possible for a given stimulus. 
Usually these are analyzed for total number 
and the nurriber. of different categories 

. generated within the associations for each 
stimulus. 

2. Construction — the learner is asked to 
produce something. Given a simple stimulus 
the learner constructs a picture or a story 
based on the stimulus. The product is then 
scored on the basis of originality, divergency 
of thinking or perhaps McCellends' Nach 
(need to achieve). 

Completion — the learner is asked to finish a 
sentence, a paragraph, or a story line. The 



result is usually ciassified by a judge and the 
results counted or profiled. 

4. Choice or ordering — the learner is usually 
asked to indicate a ranking for some 
statement based on a rating scale. In another 
approach he may be asked to choose from a 
group of statements those with which he 
agrees. The choices or rankings are usually 
summed or averaged and then treated 
mathematically. 

5. Expression — similar to construction, but 
the emphasis is not on the product but the 
way in which the product is created. The 
number of times a learner manipulates or 
perhaps approaches something becomes the 
measurement variable. 

Unfortunately the projective techniques generally 
(see 1, 2, 3, and 5 above) require a judge to make a. 
careful but subjective analysis of each response. 
This is not only inordinately time consuming but 
it also tends to be unreliable. Usually the more 
reliable the judgment, the more time consuming it 
is. Now it is possible using a relatively structured 
stimulus as in No. 4 above and using certain scales 
or scaling techniques to more reliably measure 
responses. Let us examine Table I to see which 
projective response methods are most useful for^ 
the previously specified areas of affect. 



Scales 

Examination of Table I reveals that the choice 
and/or ordering response procedure will tend to 
yield the most objective measurement data. This is 
achieved at some loss of the "projective" property 
because the items are more highly structured. 

Scales are measuring devices designed to place an 
individual on a continuum of agreement with 
regard to motivation, an attitude or a value. Three 
major types of scales are: 

i. the cumulative scale (Guttman-type) 

2.. the equal-appearing interval scale 
(Thurstone-type) 
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3. the summed rating scale (Likert-type). 

Scales then furnish a relatively effective device far 
using the choice or ordering response procedure to 
"measure" motivation, attitudes, or values. 

Both the equal-appearing interval scale and the 
cumulative scale allow the learner to respond only 



with one degree of agreement or "a forced 
choice." They do not allow intensity of expression 
of motivation, attitude or value. Tlic summed 
rating scale does on the other hand allow this. The 
advantage? It simply permits retaining of the 
maximum projective quality in a relatively 
objective measurement procedure. 



Scale Construction 

The only scale construction considered here will be the construction of a summed rating scale 
(Likert-type). ... 

One of Likert's primary concerns was the development of a- measurement device in which tfie items 
measured the same common factor (e.g., motivation, attitude, or value). To devise your own L'kert-tvpe scale 
use the following steps: 

1. Determine which attitude value or aspect of motivation you wish to "measure." 

2. Create or collect a large number of items which are either obviously favorable or obviously 
unfavorable to the factor you intend to "measure." The collection should contain only one 
kind of item, not both favorable and unfavorable. ^ 

NOTE. Tips on writing good items: (a I be clear and unambiguous; (bl be brief; (cl make sure it contains only 
one complete thought; (dj use simple sentence format, 3nd; (e) use a five point agreement continuum with the 
item — e.g., - 



Strongly agree agree uncertain disagree strongly disagree 

3. Identify a large sample of learners who are representative of the population to be measured by 
• the final scale. (The larger the sample the more reliable the resulting scale.) 

4. Give each item to each member of your sample. Total the scores for each subject using a value 
^ of 5 for strongly agree down to a 1 for strongly disagree if the attitude is favorable. ■ 

(NOTE. If the attitude Is unfavorable then use a value of 5 for strongly disagree down to a 1 for a strongly 
agree [see Step 2].) — 

5. Determine which subjects fall in the upper 25 percent of scoring (these subjects tended to 
respond with agree and strongly agree). 

6. Determine which subjects fall in the lower 25 percent of scoring (these subjects tended to 
respond with disagree and strongly disagree). 

7. Calculate the mean score for each item for the upper 25 percent. 

8. Calculate the mean score for each item for the lower 25 percent'. 
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9. Deinrmino the difference score for encfi item (Step 7 - Step 8 = diffoinnce score). Keep the 
items with the highest difference scofes. These items become the items for use on the final 
scale, 

10. When you administer the finni scale, use the same scofijig procedure as in Step 4, The score of a 
leafner is the sum of his response weights. Tliis score will rank the learner with respect to other 
learners in regard to \hQ particular factor measured. 

•A? 

NOTE, Be sure to randomly alternate the direction of the response scale, e.g, — 



•strongly agree agree uncertain disagree strongly disagree 




idis^gree disagree uncerta'n agree strongly agree 

This should be done in order to avoid response set o.n the part of the learner.) 



Now for a Caveat Emptor - — 

Be'sure that each item written is based upon a characteristic or property of the same area of affect. 
This means that in Step 2 above, you ought to identify all pertinent characteristics of the factor you 
wish to measure. Then prepare a set of items based on each characteristic. 



Summary . ^ 

By now some of the problems in measurement in the affective domain will be apparent to you. It is a 
difficult area in which to attempt measurement but it is not impossible. 

This paper has. attempted to delineate to some extent areas of affect, to specify useful response 
techniques to show problems of analysis of responses, and to show you how to create a scale for measurement 
in the affective area. This is by no means a complete discussion of measurement of affect but rather a doorway 
into this area of concern. 
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ACADRMIC CREDIT: 



KXPOSURK OR AITAINMKNT? 



J. W. Harris 

I)ir(M;tor, Inslriiclional Rc>;oiirc!(\s^ Florida Slalc Univorsii\\, Tallahai^s(*e 



There are two purposes for tliis presentation this 
morning. First, to question tl^e potency of the 
present academic accounting system in American 
universities. In working witli llie veterinary faculty 
at Georgia and others in professional education, I 
have come to feel that all of us are overly 
impressed with or committed to the present 
accounting system. This system is particularly 
crippling in tliat it is primarily based on time or 
exposure rather than stipulated competencies. 

Second, I suggest to you that tests are like 
budgets. If one wants to know what the actual 
priorities are of a school, club, or governmental 
institution, he looks at how that organization 
spends its money. The actual educational 
philosophy of a college is revealed in what and 
how it recognizes achievement. Thus the two 
major points I want to impress upon you this 
morning are: (1) The present academic accounting 
system in American academic communities is not 
good; (2) Testing, more humanely considered and 
more broadly developed and applied, can be an 
answer to some of today's problems. 



Serving Time 

Our educational practices seem to result in having 
students serve time rather than attain a set of 
definable goals. Under a Ford grant 
Hannah Kreplin did an extensive review of how 
American education got in the business of giving 
credit in terms of hours spent. This review reveals 
that from 1870 to 1898 colleges began to measure 
the teaching uf subject matter in hour units; and in 
the period of 1890 to 1910, the credit system was 
consolidated, high school and college courses were 
listed in units of credit and degrees were given on 
the-'basis of a certain number of credits earned. 



One of the primary stimuli that brought this about 
was related to the development of faculty pension 
systems. That is, the time unit approach . to 
crediting educational achievement v\/as confirmed 
in eligibility standards for institutions to 
participate in the pension program for teachers. 

Kreplin points out that the Carnegie Foundation 
for the Advancement of Teaching in 1906 initiated 
a pension plan for college teaching. Institutional 
eligibility to participate in this plan included these 
conditions: the college should have at least six 
full-time professors engaged in university and 
college level instruction; it should have a program 
of study of four full years in the liberal arts and 
sciences; and it should require all students that are 
admitted to have four years of high school, 
preparation or its academic equivalent. In 
measuring four full years of high school 
preparation, it was expected that the accepted 
high school unit of credit would be used (one 
credit as one academic year course meeting for f ivd 
periods each week). This we now know as the 
Carnegie unit. According to Kreplin, the Carnegie 
Foundation stated explicitly that the fundamental 
criterion was the amount of time spent on a 
subject, not the results obtained. The General 
Education Board of the Rockefeller Corporation, 
through its pension plan, adopted the Carnegie 
Foundation f^ormula for eligibility of institutions. 
By 1910 educational credits in liitjh school and 
college were securely based on time 
considerations. 

Bill Owens recently recommended a study to me, a 
section of which I want to read to you. 

"Colleges and schools like prisons have dealt 
with their inmates largely on the basis of 
required sentence of time to be served, A 
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collego nducation normally requires four 
years after high school. During these years a 
prescribed number of credits will be earned 
and nnch of them rnqu'res so many hours of 
study spent in laboratories, libraries and high 
schools. There are, of course, certain things 
rnquired to pass the course; but over and 
above those, ihe only measure of education 
commonly used is time. However brilliant 
the student's performance, he usually must 
still serve his hours for each credit and 
accumulate his credit over the four required 
years." 

The study involving the above quote was primarily 
concerned with an analysis of college students in 
Pennsylvania along with a representative sample of 
high school students. A six hour test that included 
such things as vocabulary, literature, general 
science, fine arts, history, and social studies was 
used to measure the academic proficiencies of the 
students. Some of the findings were as 
follows: On the English section of the test, one 
third of the freshmen from ten liberal arts colleges 
.scored higher than one half of the seniors. In 
general science, literature, fine arts, history and 
social studies, one senior in five was below the 
freshman average. In mathematics, the average 
freshman v^/as better than the average sophomore, 
the average sophomore better than the average 
junior, the average junior better than the average 
senior; only one third of the seniors were as good 
as the average freshman. Fairly expert knowledge 
of science was found in 300 of the seniors, 
250 juniors, 300 sophomores, and 80 freshmen. 
Nevertheless, the 80 freshmen who knew science 
as well as the best 300 seniors and better than 
600 other seniors still had to serve their three 
years of time before they might rate as ripe for 
graduation. 

In the case of one of the colleges, which was 
representative of the colleges, in above study, 
the question was raised — what if that college had 
graduated its 200 best students based on the test, 
rather than the 200 students that had stayed at the 
college for four years. That is, what would the 
graduating class have looked like? An analysis 
revealed that the graduating class would have been 
25 percent seniors, 28 percent juniors, 23 percent 



sophomores, and 24 percent freshmen. The 
average age of tfie graduate would have been 
reduced from 22 to 20 years, and the average 
performance on the test would have been 
increased by abuut 150 points. Thi- study was 
done by Learned jnd Wood from 1928 to 1932. It 
was reported on by Goodwin Watson in a Carnegie 
Foundation monograph in 1938. It is interesting 
to note that Goodwin Watson has been 
. particularly involved in the University Without 
Walls approach where the central concern is to 
credit students' attainment without regard to time 
spent. 

How does one explain a system of collegiate 
educational system supposedly designed to 
produce competency in students that grants 
degrees in terms of time spent. Kenneth Bolding in 
a recent article on the schooling industry in 
America,states there are four basic services of that 
industry. One is to produce knowledge in students; 
a second is to produce a certain set of skills; and 
the fourth to certify competency. However, what 
is the third? Custodial care! He makes the point 
that if a local school board wants to pass a 
referendum to increase school taxes, it would be 
judicious if the referendum were held near Labor 
Day after mothers have had their children home 
for three months. 

How did we get to a four year curriculum? 
Kreplin's Review of Credit by Examination cites a 
study by Jones and Orlmann which sets forth 
several reasons why American education settled on 
a four year curriculum. First, little trust was put in 
teachers' final examinations, and because external 
examinations were not established in. the United 
States, the time spent on the college campus 
became the convenient^measure for the accrediting 
agencies which started functioning in 1914. Such 
agencies were seeking a standard by which to 
evaluate colleges; hence they promoted the theory 
that all students should take a prescribed amount 
of work in a prescribed fashion. Furthermore, 
American educators came to emphasize four years 
of social and intellectual campus living quite apart 
from credits earned. For example, 
James Lee Peacock said that the first thing he 
would do if he started a new college would be to 
build a smoking room; and then if he had any 
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money left over, he wouJd builB* a library. I( there 
were any money led overaf^er that, he would hire 
a taculiy. It was in this irabiiion that Princeton 
had social clubs. Many of us went to small liberal 
arts colleges, some o( which had a religious flavor. 
These colleges inevitably incorporated close 
inter-personal relationships among faculty and 
students and emphasized that there was something 
to be gained beyond the explicit competencies in 
the formal curriculum. But how well can that be 
done in a university of 30,000 commuters in an 
urban setting that hasn't a single dormitory? The 
growing sector of American education is the 
community college, and the newer ones do not 
have dormitories. We have moved a Ibng way, and 
perhaps will go even further from the more 
cloistered approach to higher education. In a recent 
issue of Saturday Review, Husen reviewed a survey 
of multinational mathematics achievement of 
13'year-old students conducted by the. 
International Association for the Evaluation of 
Educational Achievement. Such factors as length 
of the school week, time devoted to homework, 
time devoted to mathematics homework, and time 
devoted to mathematics instruction accounted for 
only about three percent of the total difference of 
achievement in mathematics. Altogether this 
survey included 26 factors, four of which I 
mentioned above. The 26 factors only accounted 
for 25 percent of the variance of the students in 
their achievement on mathematics; the four that I 
cited accounted for 10 percent of the 26 percent. 
It was concluded that for both 13-year-olds and 
pre-university students, time devoted to 
instruction seemed to be insFgnificant in all of the 
12 countries included in the study. 

Data similar to this was cited to a past dean of the 
College of Education at the University of Chicago; 
trtat is. that time spent in school and the quality of 
schooling seemed to make no difference in the 
achievement of students. His response was "I've 
never thought that the quality of teaching made 
any difference." The Coleman report that is being 
systematically reviewed at Harvard and a number 
of other reports appear to indicate that the 
amount of money spent on students or the nature 
of the instructional process makes little difference 
in students' achievement. As a result, those 
interested in international education are suggesting 



that the Western European and the American 
schooling models sliould not be adopted by 
developing countries. .Rather, such countries 
should try to integrate work and learning and not 
create separate institutions lor the training of thcii 
people. 

There is in our culture an interest in 
de-institutionalized educatljii. I'm not Sure how 
far I would go in this legard, but I am very 
committed to the idea that we siiould separate 
credit from time spent in school and relate the 
giving of credit and degrees directly to the 
consenually observed achievement. Some of you 
may hestiate in that you believe that education is 
not just achievement. It may not be, but I would 
challenge all ot us to thoroughly and 
comprehensively analyze the content of the tests 
upon which we presently give credit and see how 
well they reflect the lofty curriculum goals that 
appear to be more rhetoric than reality. The fact is 
that if we were profoundly concerned with 
recognizing academic attainment, we would not be 
giving credit for passing courses when the person 
who teaches it, also tests students, and then judges 
their performance by his own standards. 

While tests have been chastised throughout the 
Sixties, they can contribute much. In the 
previously cited study of Pennsylvania colleges, 
the student that. scored the highest of all the 
students attended tne college that had the lowest 
average scorjj. The student was asked, "How do 
you explain your phenomenal score?" He replied, 
"It's due to my careful study, that I did entirely 
on my own, of every Sunday edition of the New 
York Times." This would not have been revealed 
witnout tests. 

Advocating the separation of evaluation from 
instruction is not a new idea. The University of 
London from 1836 to 1900 has no internal 
students. Its prime purpose was to examine 
students of other institutions and award them 
their degrees. Oxford has always iiad a tradition of 
having its students stand for in-depth 
comprehensive exams for their degree. The 
University of Georgia also had the tradition and 
Colter has described its practice and rationale as 
follows: 
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"Examinations were based on the 
assumption that the student had been 
amassing a sum of knowledge which tended 
to unify and coalesce into a related whole, 
not filling little compartments from 
textbooks unrelated and to be immediately 
forgotten when the crises had been 
successfully passed. Therefore the 
examinations at the close of the first two 
terms were mere harbingers of the searching 
inventory to be taken at commencement 
time on all the studies of the proceeding 
year. The senior examination was guarded 
with particular care because the Bachelor of 
Arts degree was given only to those who were 
successful. This examination was a general 
one upon all the studies of both the last two 
and many of the proceeding year. Or as the 
rules stated, it should be rigid and extend to 
the whole course of collegiate literature and 
only those found well skilled in the liberal*^ 
arts'^and sciences should be given degrees. 
Following their vacationr the students 
devoted their whole time to reviewing' their 
past four years' work against the day of their 
examination." 

The rationale was as follows according to Colter: 

"The test of the pudding Is the taste thereof 
, is a saw honored with age and truth. 
Examination time was tasting time and since 
tasting should be done more/than'*byHhe 
cook alone, in the original charter, a Board 
of Visitors was established. It first took on^ 
flesh and blood in 1811 when three trustees'^' 
were named Visitors. Later the number was 
increased to fifteen and membership was not 
limited to the trustees. The most 
distinguished men in the state were generally 
named and urged to attend the senior 
examinations as welPas the others. In 1825, 
President Waddell noted that the governor 
attended the junior, class examination. In 
1834, the University announced that the 
senior examinations would begin on June 23 
and continue from day to day until the 
Board of Visitors was satisfied, and added 
the college and faculty would be gratified by 



the attendance of parents or guardians, and 
literary gentlemen who may have it in their 
power to attend." 

The word accountability was not used by Colter, 
but the concept is incorporated by what he 
described, However, public people were invited to 
a public examination of the students before the 
state u niversity granted its degree. 

Dartmouth as early as 1831 had external 
examiners. Eliot, the great reform president at 
Harvard, advocated external examiners in his 1869 
inaugural speech at Harvard. Paradoxically, he also 
promoted the elective system in American higher 
education, and that may have done as much as 
anything to entrench the single course 
examination. 

Hutchins established a board of examiners at 
Chicago in 1831. Bloom said that the purpose for 
the board was as follows: 

"In planning a new curriculum in general 
education the faculty wishes to separate the 
examining and judging function from the 
pedogogical function. They wished to have 
the instructor serve primarily to help . 
students learn and they believed that an 
ideal student-teacher relationship was 
impossible when the teacher also had the 
responsibility for judging and grading the 
student." 

In medical education, the' strongest program, in 
" my opinion, of proficiency measurement is being 
directed by Christine McGuire at the University of 
Illinois Medical School at Chicago. As I understand 
it, professors teaching courses in the professional 
curriculum do not construct their own tests. 
Rather tests are developed by committees 
composed of professors and measurement 
specialists. Emphasis is upon problem solving items 
rather than recall of information alone. Given all 
of the theoretical merit of and a history of 
prestigious support for separating the evaluation of 
students' proficiencies from the* facilitation of 
teaching of those proficiencies, why has this not 
been done more extensively in American higher 
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education? A number* of contributing factors 
emerge as one considers this question. I will 
suggest the ones that have occurred to me but in 
no order of importance. 

One such contributing factor is the dedication of 
American colleges and universities to the elective 
system. Students tfind to have such disparate 
programs' that a common comprehensive 
examination would be difficult. Therefore, Eliot's™' 
interest in outside examiners has been thwarted by 
* his other initiative of introducing the elective 
system. Another factor is that mass higher 
education has depleted our ability to do in-depth 
critical examining of individual performance. 

But I believe that we already have strategies and 
techniques in measurement design along with the 
advar)ces in computer applications to testing that 
will make it feasible to examine the proficiencies 
of individual students in terms of the educational 
goals of their particular degree programs. 
Furthermore, veterinary medical educators dp not 
have the masses of students to deal with that 
faculties do in many other areas of a university. 
Finally, I would suggest a change of roles that^ 
faculties may need to adopt. That it is essentially 
to make students more responsible for their own 
learning given clear objectives for their study; such 
a change would make it possible for faculty to 
spend more time inspecting students' achievement 
either for placement, diagnosing learning 
problems, and eventually certifying proficiencies. 

Another factor has to do with a socio-economic 
function of education in our society. We 
sometimes naively assume that education is only 
to produce competency' when actually it Ms 
providing a way to occupy youngsters in advanced 
technologicaLcultures that do not need them for 
its work force. This was Boulding's point, as I 
cited earlier—'custodial care is one of the four 
economic functions of education in America." 

Fourth, explicit standards and judgments based on 
them tend to frighten us. Students sometimes 
would rather try to slip through the cook-tasting 
mode rather than having to confront a more open, 
objective, system. Perhaps they feel insecure and 
wish to have the possibility of "conning" the 



teacher for more than they might get from the 
more open, objective system. In one of our 
neighboring universities a science course is being 
taught. . using audio-tutorial approach. The 
enrollments in that course have declined 
somewhat. Some observers close to the situation 
believe that the students may be avoiding the more 
explicit standards of the audio-tutorial approach. 
Furthermore, some suggest that American students 
today want an emotional "experience" rather than 
intellectual mastery. If this is so, it is a 
consequence, Lbelieve, of over institutionalizing 
education in that we keep youngsters in school 
passively receiving information too long. They 
have little opportunity to apply what they learn, 
and perhaps are satiated with cognitive input; thus 
they eschew the intellectual or cognitive and turn 
to the emotional or affective. 

Some feel that more standards might lead to a new 
social elitism but this does not necessarily follow. 
The point that is being made about mastery 
learning by Bloom and others is that we want to 
set minimal standards that everyone who gets a 
particular certificate or credit should meet and not 
unduly limit that effort by time constraints. Under 
this approach, the main difference among learners 
would be how rapidly they move through a given 
system, not how much they achieve in comparison 
to one another. To me, this is a better approach' 
than having everyone spend the same amount of 
time and then having their individual differences 
manifested in different amounts learned rather 
than time spent. 

Professors in general may be jeluctant to make 
judgments about the performances of others. Ebel, 
who writes very insightfully about evaluation 
problems in higher education', has written the 
following: 

"Instructors, .-.sometimes dislike to 
assume the role of examiners. Most of them 
. prefer to be helpful rather than critical. 
There is something inconsiderate about 
probing the minds of other human beings 
and passing judgments on their 
shortcomings There is even something 
presumptious in assuming the right to set the 
standards by which others will be judged." , 



Given the fact that students do not need as much 
the delivery of information, as we seem to think 
they do, given the frequency with which we 
lecture. However, they do appear to need a great 
deal of guidance toward and certification of their 
proficiency; therefore we are going to have to 
spend more time doing what we are not so inclined 
to do — evaluating their (students) achievements. 

I am going to be audacious enough to propose a 
new role for faculty members. I suggest that the 
role of ci faculty is to become a collective referent 
of academic achievement. For example, a small 
college is trying to buitd a competency matrix for 
its curriculum. Though this is a desirable move, I 
predict that it will be hard for the students to ^ 
identify with such schema. If such matrix of 
competencies could be translated into what the 
faculty actually represents *tfien students could 
more readily understand and identify with the \ 
goals of their education. First of all, any faculty 
should be the personification of knowing some 
area of study well. There is a difference between 
just being an intelligent layman and knowing 
something in particular in depth. Furthermore, 
once a profession, such as veterinary medicine, has 
come to perceive^of itself in terms of such a 
distinction, it will only invite persons into its ^ 
fraternity who are different from the general 
population in the same ways. Every student who - 
graduates with a D.V.M. is to become your 
coljeague in the profession; and his knowledge arid~ 
competence should be . referenced against the' 
faculty's proficiency. Therefore, first a faculty 
must determine". what it knows that makes it 
different from- generally intelligent laymen and 
then learn how to inspect for these critical 
differences in its candidates for graduation. ^sjs 
a very-difficult task and you have a long way to 
go, but no further than other professional groups, 
in learning how to reliably -inspect for. the critical 
distinctions of being a veterinary medical scientist. 

Many of you may know or have heard of 
Walter Perry, the former Dean of the Medical 
School of Edinburgh, now the Vice Chancellor of 
the new Open University in England. One of the 
achievements of-ttie Open University that he is 
proudest is that, programs- of study have been and 
-are being developed by professors working on 
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teams with educational technologists. Cooperative 
teams produce systematically designed programs 
of study to reach certain goals. 

Perhaps, one caution should be observed at}out 
planning instructional programs for other people 
even when you know what the instructional goals 
are. At one university it is reported that when a 
new building is finished sidewalks are not put 
down right away. Rather they wait to see where 
the students make paths than sidewalks are put on 
those paths. The lesson for curriculum planners is 
that if they are clear about where they want 
students to go and reasonably adept at 
determining when they got there, educators could 
stand back awhile and see what kind of learning 
routes the students 'make. That is, given the 
resources of the university, which resources do. the 
students use in order to gut where they need to 
go? I hope that some of this wilt be done in the 
University of Georgia's experimental curriculum in 
veterinary medicine scheduled to begin this 
September. ' 

Virginia Zachert at the Medical. School at Augusta 
has another approach to curriculum design. She- 
has 3 professor delineate what he wants the 
student to ,know- or master. In essence she just 
says, "Tell me, what are the things that make the 
difference between a person who knows and does 
not know this area?" Then she had students that 
have mastered these same things work out a 
program of study that will help beginnirig students 
learn these things most quickly. It is hard for a 
professor to perceive learning tasks as they are 
perceived by * someone vyho is at the point of 
ignorance. On the other hand, a person totally 
ignorant of the matter to be. learned could not 
design a program of study,. So Dr.^Zachert has a 
good compromise. She has a person that had just 
learned, design the instructional program. because 
he is the best compromise between knowing how 
the untutored perceive the task and what it is that 
must be learned. 

In my opinion, the major 'task before academics is 
to begin to do a better job of evaluating student 
achievement. M. W. Richardson said- 36 years ago 
that no one haS'Solved completely the question of 
making examinations reflect course objectives' As 



far as I know we are not doing much better today. 
Progress, in this regard, can only be made by our 
becoming very clear about criteria of achievement 
.which demands either very clear instructional 
objectives or reasonable consensus in judging 
creative achievements that cannot be set forth in 
explicit terms. In regard to the latter, there are 
areas of achievement, for veterinajy educators 
such as in judging clinical performance, where you 
Will have to rely upon consensual judgmehTs rather 

• than explicit objectives. When you cannot be- as, 
verbally explicit as you would like to be^in^stating 
objectives you should concentrate on the degree of 

' reliability' of the jury or panel judgments of 
performance. 

A great - deal is being said about criterion 
measurement. If five years ago one had wanted to 
know how -to: make better tests, somebody like 
myself would probably have handed you a 
pamphlet on construction of examinations. One of 
the jhj^ngs that pamphlet would have told, you is 
th*3t after you have given the test you should 
analyze each item in terms of its discrimination 
capability. Perhaps you would have separated the 
upper 25 percent of *the students on the test from . 
the lower 25 percent. Then you would have 
identified those items that discrimnated between 
the top 25 percent and the lower 25 percent. 
Those items that perfectly discriminated would 
have had a. discrimination index of .50. These are 
the items that you would have kefjt for future use; 
the others woufd h^vebeen thrown out. Given .this 
approach, assume that you had taught 20 students 
a set of surgical techniques. Further assume that 
they knew nothing when they came in but all 
became quite proficient by the time they were 
through. It is obvious that your instructional 
program was effective in that they, all became - 
proficient. Furthermore, in the different test 
situations they all performed satisfactorily. . Given 
this, how are you going to do an item analysis in 
terms of a discrimination indices? According to 
the traditional view, you should. discard your items 
because everyone got , them. Most standardized . 
tests are built on this approach, and thai obviously 
reduces systematically the ability of tests^ to be 
sensitive to treatment, differences—that is, to 
instructional effects. 



I will close by. discussing some of what will be 
involved if you really get serious about criterion 
measurements. Several years ago Gullikenson 

. published an article on how to establish content 
validity beyond having knowledgeable people say 
that the items are appropriate; I will very briefly 
review his article. The typical way to go about 
determining if one has a good test is to .have 
experts look at the items on the test and 
determine if it is a good item. That may be about 
the best we can do at one level, but Gullikenson 
advocates that vye go beyond this in several 
empirical ways." One way is a factor analysis 
approach. He gives the example of a spelling 
test— -typically administered by > having the 

■ teacher read so mariy words with the students*, 
writing them down. A student may be able to spell 
the words under these conditions; but, if he is- 
writing a theme; he may misspell, the same words. 
Or, if you ask him to recognize correctly spelled 
words from incorrectly spelled ones, he may not 
be able to do so. Thai is, the criterion may be 
multi-factored. What needs to be done' is to 
determine if such is the case, and if there is only 
the appearance of multi-factors, and: if the factors 
are clustered. 

^The second way is related to judging performance • 
by juries or panels. .It may be that people , on 
panels are responding from different prcspectives. 
For example, you may ask a jury to judge the vyay. 
a veterinarian sets lip his practice in terms of 
management. One professor might Jook at it in 
tetms of efficiency. Another may ^iew it only in 
terms of a profit rnargin. Another may be looking 
at its public - relations effects. Therefore, it is- 
important to factor, analyze the. judgments of the 
experts on the panel. 

A third approach is. to examine the criteria in 
terms of : the variables that predict , them. For 
example/college grades are best predicted by high 
school grades\and. slightly less well by tests of 
verbal ability; College grades do hot predict 
anything very well— even grades in graduate 
school arid professional school.. Grades in\ 
professional school, according to some studies 1. 
have seen in the area of nnedicine, do not predict 
performance in the clinic much above the level of . 
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chance. Grades received in graduate school also 
seem to have • a very low relationship to the 
number of articles one may publish in his career. 
. This ought to tell us something — perhaps that 
grade getting behavior only predicts grade getting 
behavior and that not too well. In this case 
perhaps we ought to judge grades in the light of 
what they predict or fail to predict. 

Dn the other hand, judging criteria in terms of 
what predicts them may be seen in another 
example. During World War W] Gullikenson found 
.that a test of mechanical knowledge and 
comprehension was a poor predictor of final 
ratings in marine engineering school. A simple ' 
arithmetic test was the best predictor. This was 
strange in that the training involved only brie week 
of training in arithmetic and four weeks of training 
in mechanics and shopwork. In calculating final 
grades, the arithmetic score on the finaLtest was 
only one-seventh of the final, grade while the shop 
work grade was weighted four-seventh's. So why 
would the arithmetic test account for a great deal 
more of the variance of the final grades than the 
mechahical comprehension tests which was 
weighted much more heavily. An investigation 
revealed that grades of shopwork had no useful 
referents for their judgments and just tended to 
rate everyone's work about the same. These raters 
realized that 'they could. not be very reliable in 
their judgments so they got together and decided 
to Jet all ratings fall in a very constricted range. 
The solution lay in developing better 
instrumentation for evaluation. Such was 
developed; and .when it was employed, the test of 
mechanical knowledge and comprehiension became 
the best predictor of final ratings. If we are in the 
situation where verbal ability is the best predictor 
of grades in veterinary* school; and grades in 
veterinary school do not predict performance in 
the field; what we; may be doing is training 
students not necessarily veterinarians. Finally, a 
" combined factor analysis of the predictor variables 
as well as. the criterion variables should produce a 
comprehensive matrix of what constitutes the 
criteria. 

Gullikenson's last point is that we. usually judge 
the educational treatment in terms of the students' ■ 
perfprmance on the criterion test. It may be that 



we will have to judge the tests by how sensitive 
they are to educational treatments. 

In review, serving time is a. poor measure of 
educational achievement. I propose the separation 
of certification and evaluation from instruction, 
and I suggest that this is.not a new thing under the 
educational sun either in advocacy or practice. 
Faculty, in general, are reluctant to make 
judgments. People who have written very 
sympathetically about evaluation from a teacher's 
point of view say that instructors sometimes 
dislike to assume the role of examiner. Most of 
them prefer to be helpful rather than critical. 

Nevertheless, I have proposed a different role for 
faculty members. First,. what we have been doing 
as far as dispensing information appears not to 
have been very effective. Dubin and Tavegia found 
that there was no difference in students'- 
performance who had been instructed differently 
on teacher-made examinations. An instructor at 
the University of Tennessee, a very exciting 
lecturer, decided that he would see if coming to 
class made any difference in the performance of 
students. So he randomly assorted students in 
introductory psychology into: <1) a group that 
did not attend class. (He told what they would be 
tested on and disrtiissed them until they came back 
to the mid-term and then the final.) (2) Another 
group , came to class one day a week ahd^ (3)a . 
third group came three days a week. When he got 
through with the course, he could not distinguish 
the groups in terms of . their performance on the 
final examination. T believe that we are riot so 
crucially needed as frequent lecturers that we / 
cannot afford to do less lecturing so that we can - 
spend more time assessing student. achievement. 

What will happen if.creditjs not based on amount 
of. exposure. I think it wiji cause us to cfearly 
delineate our functions in education and thereby 
achieve a clearer focus of what we are doing. It 
will also make possible the individualization of 
instructi.pj[) in that tinrie will not be a major factor; 
and it will . give .you, the faculty, the ability to 
exercise some control over the quality of, the 
graduate that you certify. 
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Effective career planning for the prospective, 
student and efficient manpower planning for the 
profession are, conceptuallyrtwo sides of the same 
coin. If we use the student's best capabilities, we 
best serve the profession that he enters. Essentially 
both-bf these concerns turn about the same 
question; i.e., what does this prospective student 
need in order to be successful, first, in school and 
second, in his chosen profession. Clearly, an 
answer to this question is going to be a partial 
answer. It's going to suggest a few major 
dimensions, but that is all. Let's try to answer it. 

At least in part theri, the prospective student needs 
some of the following things. First, he needs 
riealistic knowledge regarding the nature of both 
straining for and practicing inThe profession. Now; 
I would underscore realistic here. I would say this 
for several reasons. First,' of cpurse/he has to be 
able to ieel himself in the rpje of the practitioner 
and it's important that the information be 
accurate if he's going to do this well. The literature 
of both vocational guidance and industrial 
psychology clearly indicate that realistic 
expectations reduce turnover. In industry this sort 
of thing has been approached along the lines that 
you tell a job applicant not only what is good 
about the job that he's applying for, but what is 
bad about it and what jt is that he probably won't 
like. And: if he takes .it under these circumstances, 
he's Ijkely to stick. If he takes a description of the 
job that's been carefully washed with igold; he may 
find, unddubtedty will find/} that it-snol up to his 
expectations and he may very well drop it. j think* 
that realistic is a very important word here. - 

Second, Or. Harris' proposal that we advance and 
graduate students on the basis of competencies 
makes great good sense, if we know or can learn 



the relevant skills and competencies. SO:here also 
we need realistic knowledge of the profession. I 
want to suggest that we might get a little feedback 
from the field, in terms of a questionnaire or 
opinionnaire. Clearly, we^also need a very careful 
job analysis. No opinionnaire is going to be; 
adequate for this purpose. Third, * a rapidly 
changing field, which I'm convinced veterinary 
medicine is, needs benchmarks. in terms of which 
to measure its evolution or in terms of which to 
porject its future. 

Let me read four things out of a little piece that's 
done by Richard Holdeman at Purdue which have 
to do with getting some feedback from the 
profession. I thinks all of the points that he 

- mentions will be familiar,.but I think all of them 
are well made. Among them, Holdeman's results 
show that veterinarians believe that their 
profession is taking a turn toward specialization, 
and they feel that educationaV revisions must be 
made to accommodate this trend. That is, they 
feel that some allowances should be made to 
permit students to specialize or choose electives 

.while in veterinary medical school. Second, course 
areas which; were specified most often as in.need 
of more emphasis'include business management, 
liberal arts, nutrition, preventive medicine, arid 
diagnosis Many veterinarians stressed the need for - 
more practical experience Mjuring veterinary 
training, and some recommended that internships 
be required. It vvas felt that future recruitment of 
OVM's should take into account changes in the 
nature^ and directions of growth within the 
profession which imply that the necessity of a 
rural background, for example, is decreasing. 
Attempts should, thus, be made to give urban, 
students greater exposure to the nature of 
opportunities in veterinary medicine. 



All the first slide is intending to do is suggest that, 
graduates of veterinary schools fall into a variety 
of specjalizations under research, training, 
teaching,, administration, large animal practice,^ 
small animal practice, and probably other pursuits 
that .1 am unaware of. This is the world of 
occupational experience. It. is the world that we 
need to know something about. Some important 
questions are: In channeling students into careers, 
. what is the image .of veterinary medicine by high 
school students and their counselors?. For 
example, in the Holdeman survey, there were two 
rather surprising findings. High school students 
generally grossly underestimated the income of 
veterinarians and overestimated the educational 
requirements. Secorid, how accurate are the 
-expectations and how relevant are the motivations 
of p re-veterinary students? And third, and 
undoubtedly most important, what are the 
impressions of those in the iield regarding 
appropriate training patterns? 

A second requirement beyond realistic knowledge 
of the profession is, of course, that the students 
possess some relevant aptitude and academic skills. 
Slide two is a probability table, an empirical 
probabilityjable,. based on experience with the old 
veterinary aptitude test prior to the revision. It 
shows in the .highest ,quarter on the test the 
probability of being in the upper half as far as 
grade point average is concerned is 0.76. .WitK 
persons in the second quarter 0.58, with people in 
the third quarter, 0.40, and persons in the lower 
quarter, 0.22.' Theh there is correspondence 
generally and broadly.:::between scoring on ^ the- 
Veterinary Aptitude Test and grades in veterinary 
school. I donlt think anybody doubts this. . 
Although we were slow to revise the ^ost/ il is 
better. now and the relationships are higher than 
they used to be. 

On slide number three, the distributions in red are 
the distributions of students admitted to medical 
schools in this case, because we don't have this on 
veterinariaris. The dotted lines are School C and 
the solid lines are School A. This is meant to point 
out that there are ' enormous institutional 
differences. You'll notice that the mean score of 
persons admitted to School A is. about 660. And 
the mean of those rejected is about 557. And the 



mean score at School C of those accepted is 483 
and the mean of those rejected is 473. Now, in 
terms of career planning and in terms of the 
profession, I'm sure the implications of this are 
entirely clear to you. First, any procedure that 
permits this sort of thirtg to happen is grossly 
unfair to the students who applied at School A 
and were rejected. They have all the potential to. 
become better veterinarians and better students 
than most of those that were admitted to 
School C. From the student's point of view from 
the career planning side, one should first inquire 
whether the student has the aptitude to graduate 
from a particular institution. It is cl'^ar that 
sutdents who may fail to be admitted to School A 
could easily get into School C. Indeed, they may 
be the best that School C has. Inequality of 
standards then represents unfairness to the student 
and a vwste of talent to the profession, it is 
accompanied by parallel inequalities of graduates. 
The reason Harvard graduates students with 
superior atptitudes and achievement test scores is 
that it admits students with^superior aptitudes and 
achievement; test scores. There isn't any 
differential effect in institutions. • 

A third requirement then when you look at career 
planning requirements and what it takes for a 
student to get into the profession is an appropriate 
pattern of interests. There may be some question 
about what interests are you for. I know it's old^ 
hat with all of you that you need an apptopriate 
pattern to go through. The chief correlate , of 
interest; however/ is hot productivity. That's not 
true in industry and it is not true on the academic 

•scene. The chief correlate of interest is tenure. 
People who are interested in the work they're 
doing stay in it/those who are too disinterested 
quit. But it is not necessarily true that those who 
are most interested produce most. They may^like 
the job so well that they go out and drink coffee 
with the boysall morning and tell each other what 
a great job it. js. Slide four shows data for dental , 
schools. These data relate to dental scores and 
continuing dental, school. They go back to the 
time when there was considerable attrition in 
order to make the point. I'm sure you can glance 

^ at this table and catch its importance as well as I 
can point it out. But among people with dental 
interest ratings of A, notice that about 



100 percent of the first year students are there and 
about 90 percent of the fifth year ^students are 
there. Now if you drop clear down to C ratings, 
look what happens. Now most of the first year 
students are still present. But look at the fifth year 
students. There are 25-30 percent of them at the 
most. The rest are gone. So the people who have 
low interest ratings tended to drop out and the 
same thing will happen in the profession. So 
interest isn't just something that is nice to have^of 
that leads to high productivity. It is something 
that keeps people at the particular job that they 
have the interest in. 

A final point and I am through. The third thing 
that we all recognize a student needs is a 
qualifying set of personal characteristics. This is 
very difficult to decide. We've played with 
psychological questionnaires and gotten nowhere. 
Rpcently at the university here, we, ha\^ engaged 
in a project which involves the administration of a 
biographical questionnaire to ent§ring freshmen 
and, cutting the facts to the bare bones, we've cast 
people under subgroupings that have about the 
same kind of background. And we find that these 
people who have the same kinds of prior 
^experiences tend to exhibit the same kind of 
subsequent experiences and that,- indeed, these 
' turn.out to be fairly good functional definitions of 
kinds of people.^ 



I'd like to try to convey two things to you. The 
first is that there is a great deal of evidence that 
both in the school and in the occupation that 
those people in these differing subgroupings have 
affinity for different kinds of performance. For 
example, we took this ajDproach to Humble Oil 
Company and got their'management people to 
complete the questionnaire. What we found was 
about 83 percent of their salesmen on the 
validation staff all came from three subgroups and 



the differences between the three subgroups were 
stylistic differences. The people in the one 
subgroup were mainly ex-Southwest Conference 
athletes who scored more because everybody liked 
them, they knew them and they had visibility, and 
so on. The second were a typical group of people 
of a little above average academic aptitude, a 
history of electional leadership, positions in some 
outstanding verbal pursuit. And the third group 
were people who rang more doorbells than 
anybody else. They had moved to sell. 

All I want to suggest to you along this line is that I 
haven't tried it, but I have every confidence that 
there are differences between veterinarians who go 
into research and administration and small animal 
practice and if there are any, large animal practice 
and so on. We can find them if we want to. I've 
heard, I don't know how many veterinarians, 
deans c. the veterinary schools, people I've 
respected highly say, "Well all this stuff you 
measure is good, but if you'll just^get a measure of 
motivation, then we'd be somewhere!" I believe 
that we are approaching a functional measure of 
motivation in this biographical data approach. 
We've had.no luck trying to measure motivation 
per se as an entity,' but what you measure when 
•you measure autobiographical data correlates is 
evidence that it has existed in the past in the 
individual. And the interest that it vyill exist in the, 
future then is a very good interest. 

I've tried to suggest three or four things to you 
hastily: (1) That it is important that the student 
have a realistic knowledge of the profession. (2) It 
is important that he have an appropriate set of 
aptitudes and that he apply these at an appropriate 
institution. (3) That he have an appropriate set of 
interests. (4) That he; have a qualifying set of 
personalcharacteristics. 
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John C row! writing in the March 1, 1971 issue of 
The Chronicle of Higher Education under th^ 
headline /'State Colleges Face Austerity, 
Accountability" makes the following statement: 

. In most of the 48 state legislatures holding 
regular sessions this year, two trends 
affecting higher education appear^ to. be 
taking shape: Funds for fiscal 1971-72 will 
be extremely tight, and the legislatures 
intend to keep a close watch on how 
institutions spend the money that is . 
.appropriated. 

In response to a survey by The Chronicle of 
Higher Education about legislative prospects 
for the coming year, statewide boards of 
higher education frequently used the terms 
' -^'austerity" and "more accountability." 

America's colleges and universities face fiscal 
pressures from which there is no escape save 
thorough and widespread reform to achieve 
increased productivity. Accountability is the 
policy demand to objectively relate inputs to 
outputs. In terms of resources allocated, the 
American system of higher education is the most 
expensive in the world. Given 'the "Baumol 
Crunch/'1 it is hardly surprising that sharp 
questioning now attends its productivity. 

To meet the demands of accountability — to . 
relate resources to learning and 
accomplishment — colleges must reform: (1) the. 
manner in which they deliver educational benefits; . 
(2) the traditions which undergird their purposes; 
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(3) the leadership styles and competencies of those 
who manage; and (4) the manner in which they are 
evaluated and subsequently improved. 

And they must accomplish this reform in- an 
enterprise which: 

1. spends a shockingly inadequate less than 
one-third of one percent of its budgets for 
research and development; 

2. does not use the research and development it 
has; 

3. lacks a technology of instruction; 

4. lacks an orderly means of cooperation in 
behalf of its clients; and 

5. has few effective mechanisms for productive 
discussion and negotiation with its 

'^personnel. 



Elsewhere I have discussed some strategies to 
accomplish this reform.2 In general it will take the 
adoption of clear accountability policies, the 
setting aside of developmental capital to be 
administered through grants-management, the use 
of limited and pin-pointed performance 
contracting with staff, the use of professionally 
^sound independent reviewers of accomplishment, 
and the employment of know-how from the 
rapidly growing field of educational systems 
technology. 



Educational accountability in policy form 
demands a student-centered approach 3 It spRcifies 
the forfnulation of educational goals and 
objectives which serve both as guides for action 
and guidelines for measuring progress toward 
satisfactory implementation of the goals and 
objectives. It insists upon the responsibility of 
departments and colleges for providing educational 
benefits for every student accepted for-etirollment 
and for preventing failure. It provides for objective 
reports on the performance of students and of 
units of faculty and' on progress of the college 
toward fulfilling its institutional purposes/And it 
also provides for experimentation with new ' 
techniques for systematic improvement and 
discarding of ineffective and inefficient processes. 

Under the accountability aegis, learning is kept in 
primary focus. While courses are needed,. they are 
seen as sub-systems, groupings of components 
thought to effectively and efficiently achieve 
desired and clearly communicated objectives 
consistent with and derived from institutional 
purposes. 

Learning is an active process, To optimize it, the 
student nriust interact with teachers and materials, 
other students, etc. Present practice" in too many 
colleges reveals that in the classroom students 
actually spend very^ little time actively ^ 
learning — some evidence vvould put-the figure as 
low as two percent. Where this occurs, it leads to 
enormous waste when one considers:^ the number 
of students, the annual costs/ the time, spent, and 
the number of unsuccessful learners. 

In some classrooms, the only person who is active 
more>than a fraction of the time is the teacher, 
engaged primarily in: preparing and delivering 
lectures to students whose motivation for paying 
attention may be insufficient, and preparing, 
administering"" and grading tests and other 
assignments which, because of the methods 
applied and/or the primitive instructional 
technology used, have little value in helping the 
students to learn, of. the teachers to judge their 
own effectiveness. This ineffective and inefficient 
process can be carried out in expensive facilities in 
which during large amounts of time: lecture halls 
are. empty, laboratory equipment is unused, and 
libraries are used spaVingly. 



The reform needed Is possible only through a 
faculty and administrative agreement to change 
froni group-paced, time-contingent learning to a 
personalized performance or results-contingent 
learning. This means: (a) learning material 
I organized in .units that can be made available in 
alternate versions to meet individual needs or 
learning styles and that can be easily updated to 
incorporate innovations in leaching technique and 
not knowledge; (b) individual contact by each 
student with teachers and peers as well as small 
group encounters; (c) a supportive educational 
logistics; and (d) a faculty with demonstrated 
competence to perform teaching activities having 
substantial promise to pptimize learning in 
students. 



A college or university may have many 
roles — research, community service, professional 
training, etc. Considermg that the central aim is to 
facilitate learning in students, everything else that 
an institution does in the name of education which, 
is not directly supportive of this goal needs close 
scrutiny and justification. 



College personnel are accountable for: 

1 . Jhe provision of a rewarding experience for a 

student who asks questions and discovers 
answers; 

2. the^stimulation of each student to determine 
and solve learning problems with increasing 
independence; i, 

3. the provision of an environment that 
facilitates the process of learning; 

4. the prescription of learning activities best 
suited to each student's entering repertoire; 

5. the provision ^f instruction leading to 
accomplishment of objectives, using media 
and modes suited to individual and preferred 
learning styles; 

6. the . use of positive motivation through 
scheduling of desirable events that are 
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contingent upon satisfactory completion of 
objectives; 

7. the use ol evaluation and feedback at 
frequent — performance-based 'intervals 

^ permitting direct guidance to student and 
teacher; 

8. the provision of special supplementary 
and/or remedial activities to strengthen the 
learning process. 

College presidents, deans, teachers, regents, and 
others interested in furthering increased 
productivity in higher education might consider 
the following seven-point action program: 

ir Adopt' and implement specific 
accountability policies, i.e., policies that 
stress unit and system results that can be 
independently reviewed and reported. 

2. Manage systematic educational process to 
get good practice into prevailing practice in 
the classrooms and learning centers. 

3. Set aside money as development capital to 
"'serve as educational R & D money., 

4. Provide rewards for high priority results. 

5. Institute a collegial quality control 
procedure for all programs. 

6. Use performance contracting with staff as 
part of the R&D program on a turnkey 
basis. 

7. Involve the staff and students, in the design 
and implementation of the productivity 
program. 

Getting more and better results for a reasonably 
increased amount of dollars over the next several 
years is i/ie central requirement for higher 
education. This can be achieved only through, 
reform in management, delivery of benefits, and 
basic operating philosophy. Accountability is the 
collection of policy declarations to facilitate those 
reforms. 



Faculty can strengthen their economic and 
professional positions by aggressively leading 
efforts to meliorate the following six conditions in 
ways which are listed. For each condition, the 
leadership required is outlined. 

Condition 1. Within the university community, 
there is widespread misunderstanding of the nature 
of the learning process, how learning takes place 
and what is actually learned by the students. 

Leadership required: 

1. to help students clarify their own objectives 
and to link- them to, learning activities 
provided 

2. to have students share responsibility for 
defining instructional objectives 

3. to encourage students to have a voice in 
decisions about the teaching methods used 

4. to have students participate in the evaluation 
of their learning experiences 

5. to have students participate in the decisions 
controlling their university life as well as the 
change process in the university 

6. to design effective learning systems: 

(a) to define objectives 

(b) to select from? among a variety of 
approaches or combinations of 
approaches to meet the objectives of 
the course and the students 

(c) to use criteria of effectiveness which: 

(1) provide for knowledge of 
results 

(2) permit individualization 

(3) encourage active responding 

(4) optimize costs 

(5) control time 

(6) enhance availability 

7. to become, in part, a director of learning 
using the services of a great number of 
persons who may not be normally classed as 
"teachers" e.g., researchers, technicians, 
artists, members of the^ professions, 
businessmen, etc. 
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8. to use and welcome students on a 
continuous education basis. 

Xondition 2. University students, as a general 
rule, come from secondary schools which are 
teacher and course centered As a result many 
students are poorly prepared meaningfully and 
productively to participate in independent modes 
of learning (yet they^are expected to have these 
"studentship" qualities). Independent learning 
modes require special skills and experience if they 
are to be effectively used, and if students are to 
benefit from participating in them. 

Leadership required: Amona the effective (and 
validated) requirements for training students in the 
special skills and in offering experiences required 
for independent modes of learning are: 

1. how to derive objectives jhd establish 
priorities among them 

2. how to participate effectively in group ' 
situations, this requires — 

(a) an understanding of group discussion 
techniques 

(b) an understanding of group dynamics 

(c) an understanding of human relations 

3. how to analyze and solve problems using 
scientific and analytic approaches 

4. how to utilize most effectively a wide range 
of teaching devices from textbooks to 
computers. 

In addition to these more general skills, students 
must acquire specialized skills associated with 
particular disciplines such as skills in statistics, 
experimented design, rapid reading, etc. 

These general and specific comper«nc/e5 should be 
warranted. The essential point is that students 
must be prepared to utilize new techniques and 
that there are specialized skills associated with the 
learning process itself which students must be 
encouraged to develop. Teachers must not assume 
that students already possess such* skills before 
they have been clearly demonstrated, 



One additional consideration, a new kind of 
student is now entering thtJ university. In some 
cases, as a consequence of the introduction of 
modern educational techniques at the secondary 
(and even elementary! leve) these students wilf be 
better prepared to participate meaningfully in new 
approaches than is the present generation of 
students. 

Condition 3. In many cases university teachers 
fail to create systems which relate what students 
are expected to learn to their personal and/or 
social objectives. 

Leadership required: 



1 . ' to relate the university to the society 

2. to help students clarify the nature of the ties 
between what they are expected to learn and 
the world around thern.^^^ 

Condition 4. Many university teachers use 
negative reinforcement and the zero-sum game in " 
the form of marking on the normal curve. 

Leadership required: 

1. to emphasize the positive aspects of 
motivation by 

(a) insuring that the methods of teaching 
are themselves intrinsically rewarding 

i experiences * 

(b) using the constructive* 
information-feedback aspects of 
evaluation rather than its negative, 
personal and competitive aspects 

(c) redefining aptitude from a time base 
to a competency base 

(d) instituting a zero-reject program with 
the 90/90 objectives-based aspects of 
systematic instruction as a standard. 

Condition 5. Although university education is 
becoming more and more complex, preparation 
for university teaching is still largely a matter of 
acquiring expertise in subject matter. University 
teaching is perhaps the only profession which has 
not up to now provided formal training prior to 
practice. 
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Leadership required: 

1. to develop systematic approaches to the 
— professionalization of teachers and other 

staff involved in university education for the 
purpose of improving present educational 
systems and to tetter prepare teachers for 
Jutu re systems 

2. to establish -training programs using such 
techniques as micro-teaching, in-service 
seminars, "sabbatical" training leaves, etc. 

3. to provide university teachers with 
appropriate models designed to' exemplify 
the best pedagogical procedures 

4. to utilize adequate incentives and support to 
encourage faculty to become* professionally 
trained as instructors. 

Condition 6. Most college and university teachers 
do not see themselves as potential change agents in 
the improvement of instruction. Perhaps it is in 
part a feeling that the overall organizational 
structure is an impediment. 

Leadership required: 

1. to implement the principles of effective 
instruction in his own courses regardless of 
the overall organization structures 

2. to explore and experiment with new learning 
modelsand instructional techniques 

3. to create organizational structures and 
strategies to motivate and facilitate faculty 



' to improve instruction on a continuous 
basis. Mechanisms to accomplish this 
include: 

(a) systematic and regular student 
feedback 

(b) financial support for educational 
experiments and developments 
(Educational R&D with development 
capital) 

(c) . the service of experts (technical 

assistance where needed) 

(d) rewards for effective instruction (the 
university career structure has to be 
oriented towards teaching as well). 



Summary, Americans are a performance-minded 
and results-oriented people. Whether the matter be 
sports, transportation, plumbing, or denta! care, 
Americans are dissatisfied with poor performance. 
Until the most recent past, educational 
performance was thought to reside almost solely in 
educational input — -in teachers, facilities, student 
aptitude, and the like. Now it is turning to include 
student accomplishment provided by that input. 
Presently students are accountable {as are 
taxpayers and patrons), but colleges as systems are 
largely independent of both quality control and 
accounting for performance. The price of 
maintaining this status-quo is continuing loss of 
support and accountability on the part of the 
clients and the financial supporters. 

Burke's warning that "a nation without means of 
"^reform is without means of survival" is applicable 
virtually intact to the situation " in higher 
education. Only the substitution of the word 
institution for nation is required. 
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OPEN FORUM 

Presiding: C. F. Reed, D.V.M. 

Associate Dean, College of Veterinary Medicine 
Michigan State University, East Lansing 



If grades do not mean anything, why give 
examinations? 

(Dr. Harris) I feel the problem is that grades 
are typically normative reference. They are 
derived from teacher-made exams. We all 
make up exams alike two or three nights 
before or even the night before 
administration. 'We simpfy do not have or 
use good criterion measures, and because of 
this, we do not have good assessment of 
student performance. That does not mean 
we cannot obtain better measurement of 
student ability. The first order of business to 
improving instruction is to develop a better 
criterion measure. You are not ready to send 
people out to practice veterinary medicine 
without having some idea of what they can 
do. I think we are well aware of the fact that 
grades do not predict very much about what 
an individual can do in the field. The 
dilemma is not necessarily mine, it is ours. 
You have got to be able to certify to the 
public that the student can reach a minimum 
level of performance and he knows that the 
grades we are now giving in professional 
school do not do that. 



Theretf||Pfcn an effort in some cases to do 
away with professional certifying exams, 
medical boards, and bar exams, that make 
graduation from an accredited school the 
only means of being admitted to the 
profession. To me that would be a loss 
rather than a gain. By improving these 
examinations systematically, better things 
can develop. 



If we shoot for 100 percent mastery, some 
students will be in a professional curriculum 
well beyond the present four years. How 
would you handle this physically and 
financially? Perhaps you may propose that 
students go at their own rates within the 
limit of the standard curriculum length. 

(Dr. Harris) That problem has been worked 
on by Christiene McQuire at the'University 
of Georgia. There are some examinations on 
which they only expect the students to get 
the right answer only 20 percent of the time. 
They have had board certified physicians 
take the examinations and they just do not 
do any better than that. The arbitrary level 
of 100 percent or 85 percent has become 
unrealistic for some examinations. The point 
is to establish the criterion measures and 
then evaluate how well the students do. Now 
don't go overboard! The important thing is 
to develop criterion measures as a minimum 
standard first. Then try it out on both 
acknowledged expert^^and average students. 
Then set realistic levels. For example, you 
could set a criterion level so high that we 
would not have any/veterinarians for the 
next five years. WeMhe public would rather 
have some veterinarians that are lower level 
performers than none. Thus, the idea of 
absolute performa"hce has got to be 
moderately modified by normative reference 
material. 



Does accountability for each student mean 
that every student will reach the goal? 
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A. (Dr. Lessinger) I have stressed every student 
accepted, that is where morality enters. The 
thing that makes me uncomfortable is that 
afte' we go through an elaborate screening 
process, and accept students, we defeat 
ourselves by having arbitrary high quality 
standards in which only 10 percent or 
20 percent survive. This is really hedging 
your bet. If your screening procedures are 
adequate and you admit students with the 
highest aptitude, then in what will you pride 
yourself, graduating students who really do 
not need you. I make a distinction between 
that and the public school system, t think it 
is only fair to say that what might be 
appropriate at one age level is not at 
another. The accountability movement 
started out of concern about the poor, the 
black, the students who simply were not 
getting any benefits from the system. In 
Georgia, 10 percent of our first graders fail 
and I have had teachers who wax eloquent 
about how this is a good thing, that failing 
experience gives the student something. 
(Except for them I think.) ( cannot conceive 
failure in the case of a first grader — what is 
it that they do? Not color the flower petal 
design correctly. Is this failure? To me, it is 
very poor criteria, a very bad moral 
commitment on the part of the teaching 
profession. In talking about graduates of 
professional education, I would submit that, 
"everyone should be a winner" within the 
standards set by the profession. A rule of 
thumb practiced by the Armed Forces and 
industry is to develop programs that give 
you 90 percent of the people that you 
accept achieving 90 percent of the objectives 
you set. Now that may be a little too high 
for you, but it is a good target to shoot for. 



Q. Yesterday you began sayi. ' chat we should 
begin listing characteristics of a good 
veterinarian. Give us one suggestion on how" 
we might get started. What would be one 
characteristic of a good veterinarian? 

A. (Dr. Gardner) Well, as I said rather 
facetiously yesterday, in defining a good 



veterinarian would it be that the dog smiled 
when you gave him a shot. I think what you 
have to do is really look at the behavior of 
practicing veterinarians. As Dr. Harris, 
suggested, I think you have to go to the top 
quality veterinarians, observe them, talk 
with them, and note what they do that leads 
to their success. Note the way that they 
interact with the client, how they handle the 
animal. For example, a veterinarian friend of 
mine greets my dog with, "Hi, Hans." He 
had memorized his name. I love him for it. 
Now to me this is a characteristic that is very 
important and it ought to be encouraged. 
There are large numbers of other 
characteristics. You do not want to graduate 
hostile professionals. What manifestations of 
hostility are there? Turn it around and 
state: what do we want the student to do? 
This will lead you to a behavioral objective 
that can be tested. 



Q. How was the veterinary aptitude test devised 
and what correlation is there between 
performance on the veterinary aptitude test 
and the National Board exam? 

A. (Dr. Owens) The tests were devised by giving 
a sample of veterinary students the old 
medical aptitude test initially and selecting 
from it th" .ubsections that correlated with 
success in ^/eterinary medicine. A second 
step was devised by correlating course grades 
in pre-veterinary work with success in 
veterinary medicine and picking out the 
pre-veterinary courses which correlated most 
with success in veterinary medicine. Thus 
subjects like chemistry and zoology are 
included with the achievement section of the 
test and the ability to read and understand 
content are listed in the aptitude section of 
the test. 

If you ask me for evidence that people who 
score high on this test are, in fact, better 
veterinarians, then you ask me a question 
which I or no one else can answer. You are 
going to have to define what is a good 
veterinarian. This is an extremely difficult 
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task. One possibility is to define it in terms 
of tenure. Another in terms of service and 
reputation. Maybe you want to put all these 
together. When you get into the field and 
start making job analyses, you are going to 
have to know what the good veterinarian can 
do or will do, that the poor one cannot or 
will not do. This job analysis can take several 
gorms. When you are through, you may be 
able to say something about what is an 
effective practitioner. If somebody will tell 
me what a good veterinarian is I will be glad 
to relate the test to it. Until that time we 
will have to relate it to success in veterinary 
school. 

(Dr. Reed) Dr. Harris, I wasjnterested in 
your comment about serving time and the 
effect this evidently has upon students later ^ 
in life concerning attitude toward the 
educational process. Several years ago we 
conducted a survey on how many adults 
were interested in eclucation after they 
finished their terminal program. The results 
were: somewhere between 15 to 20 percent 
were interested in using such things as 
organized continuing education programs. If 
we keep track of people who attend 
organized continuing education programs, 
we find that this is about 
1 5 percent-20 percent also. Now this is not 
to say that others are not continuing their 
education in other ways, but only 
1 5 percent-20 percent are attending 
programs similar to their previous experience 
in education. 



Q. Are the State Boards of Examiners fulfilling 
the purpose of external examiners in 
professional schools? 

A. (Dr. Harris) I really do not know. I do think 
there is a general fault in this culture as a 
whole. We have got to move more toward 
performance and situation tests rather than 
our complete reliance on paper and pencil 
tests. It is going to take a massive amount of 
development capita! to get it done. 
Performance on aptitude tests predict grades 



in veterinary schools because grades are 
contingent- on doing well on paper and 
pencil tests. But getting those good grades 
does not seem to tell us much about how 
effective the veterinarian will be in the field. 
To assess this we have to move to more 
clinical situational tests. I would say that 
from what I know about external 
examinations, they are generally deficient in 
this area. 



Q, What is the better grading system, letter 
grades or pass/fail? Do you feel there is a 
greater tendency toward mediocrity of 
students on a pass/fail system of grading? 

A. (Dr. Gardner) If you can accept the notion 
of performance criteria as a basis for 
providing feedback tc the students, so that 
' the student knows how effective he is and to 
what extent he is moving toward the 
expected outcome, then I do not think that 
pass/fail promotes mediocrity at all. In fact 
it leads toward something that I think we 
value very highly in our country and that is 
cooperation, not severe competition. When 
you specify objectives very clearly, you are 
not playing games with your student. When I 
achieve the objective, you are 'completely 
immoral if you fail me. This cannot lead to 
mediocrity and the students know and you 
know the performance level by the standards 
that have been established. 

(Dr. Lessinger) I would like to comment. I 
think this is really the heart. We are dealing 
with mindset. You will recall a hundred 
years ago when they first attempted to get 
physicians to wash their hands after 
discovering that they were in large part 
contributing to the demise of their women 
patients'ln childbirth. They had an awful 
time, because" they were touching at the 
heart of the matter which is a change in 
mindset. 

It is very easy for me to step aside and 
chastise you and pretend that 1 am not a 
part of the problem. We all are. We all have a 
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mindset that Is very powerful. You will find 
yourself and I find myself constantly 
worried about this notion of the screening 
and sorting. We really have an educational 
tradition that starts in the very first grade:- 
and continues all the way through, which is 
that somebody has to be the winner and 
somebody the Icser. 

The new approach changes this basic set and 
redefines aptitude. It flies in the face of the 
idea that you cannot make a silk purse out 
of a sow's ear. That is not true. Ycu can. It 
is a matter of cost and benefits. It is 
economics that is influencing the decision. If 
you want to spend time and enough 
resources, you could move virtually 
everyone from one point to another. Now it 
may be that this is not a sensible solution. 
Society has all sorts of opportunities, but 
that is quite different from putting the 
accountability on the student. I just wanted 
to stress that this is a radical and new basic 
* definition tearing up the old roots. This says 
that when you accept a studept, and work 
with him long enough and have developed 
the right systems, he will succeed. Obviously 
reality is there. There are costs and benefits. 
For some students it is just too long and 
maybe they should do something else. This 
is quite different from talking about some 
students being good and some being bad. In 
fact there is some research to show that if 
you proctor some of our "good students" as 
to learning and forgetting, maybe the slow, 
dumb bunny may actually have better 
learning. 



Q. If everybody can learn, why not randomly 
select our students that are entering into the 
school? 

A. (Dr. Lessinger) I am not trying to fly in the 
face of interest or economics. You may have 
read in Time Magazine of the mongoloid 
idiot who was learning poetry. There is vast 
promise in the human being. We have got 
such fantastic promise ju5t because we are 
living and alive and can learn in so many 



ways. Interest, speed of learning, and the 
individual learning style are quite different 
from dividing up students on some unitary 
scale telling the good guys from the 
ding-a-lings. For reasons that are obvious in 
terms of the allocation of resources, it is 
• important to do some screening and to find 
those who are interested and have the 
attitude and aptitude. But I want to stress, 
once you say, "Yep, you're in/' then to me 
there is a moral obligation. You are 
accountable to that student, or else you 
should not allow him in your program. 
Otherwise it seems to me the educational 
profession is just not an accountable 
profession. 

Q. According to what we have learned the last 
three days, a course evaluation passed out to 
the students at the end of the course 
complied by the instructor is a red herring. 
Evaluation should be taken from a master 
student, but have we eliminated the 
necessity of a handout evaluation? 

A. (Dr. Harris) Obviously, the more direct the 
better evaluation is of the teacher. How 
much do the students learn out of the 
objectives that have been publicly stated, 
and that the students and his colleagues 
understand as well? That is the most direct 
measure and if we approach that, there will 
be less necessity of placing so much weight 
on students' opinions of the teacher. There 
are a number of studies that indicate that 
the atmosphere under which students study 
has a lot to do with the attitude toward the 
discipline itself. Let us assume that one 
teacher gets an equivalent set of students to 
accomplish the same mastery goals as 
another one does, but the second group got 
there by being pushed, shoved, and cajoled, 
everything is very unpleasant for them. 
Everything we know about learning, animal 
as well as human is: If you learn under 
pleasant circumstances, you avoid confusion. 

Your eaiiier point was that only 20 percent 
of the people want to come back to learn in 
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a similar manner to what they learned. In 
the future, moving toward a more direct 
assessment of student learning, can be a 
primary way of rating the teachers. If not as 
important, at least a close second, is the way 
he helps you get there. The extent that it is 
positive, pleasant, and so forth, there will be 
, a greater encouragement for students to 
come back. To illustrate, I do not like 
libraries. I have spent a lot of time in 
libraries, but today I just do not go to 
libraries. Now that is a bad admission. I have 
said that to some librarians and it bothers 
them. Many of you may like to spend a 
whole day in a library, but I bet not all do, 
and if you have graduate students, you will 
send them to the library before you will go. 
I think that is where student evaluation will 
come in, in teaching. 

Q. What are some references, books and so 
forth, on career development and planning 
requirements particularly to the choosing of 
a job within the veterinary profession, small 
animal medicine, large animal medicine, 
academia, and administration? 

A. (Dr. Owens) I cannot really give you these 
off the top of my head, but I will look them 
up for you and put them as an addendum to 
the brief notes that I have for your 
proceedings. I really doubt that there is 
anything real specific about your profession 
outside the literature of your profession. 

Q. Has not the cooperative extension service of 
the land grant universities been a university 
without walls, no degrees given and 
integrating learning and working together? 

A. (Dr. Harris) That is one of the best examples 
of learning in the context of need to know 
and problem solving. It is certainly where it 
is worked best. I think we are going beyond 
that now. We are going to be willing to give 
people academic or professional recognition 
if they can demonstrate the competencies 
that compose the constellation of whatever 



that profession is. I am working with a group 
on the possibility of regional examining 
universities that would examine people in all 
post secondary experiences and give them 
credit and even degrees for anything they 
do. The state of New York has its college 
proficiency program that is operating and 
doing welt. The University Without Walls at 
Antioch is another example of this. The idea 
that information is richly dispersed in the 
culture today and that the need to continue 
to learn is so strong that you credit people 
for their competencies without concern for 
their sitting on campuses and getting it. 
Some people may need to sit to get it and 
some people may get it alone. But how they 
get there is unimportant as long as you are 
sure that they got there. 



Q. What is the proposed reward to a teacher for 
being more effective in carrying out his 
responsibilities? 

A. (Dr. Lessinger) I am going to give you a 
simple minded answer because I do not 
know a complex one. One critic accused this 
of being cynical, but it is not. You can carry 
this too far, and that is why I try to balance 
what I say. 1 think that you are talking 
about the chain strategy. You are really 
talking about developing consequences to 
behavior. There is no question -that when 
you talk about accountability, you have got 
to talk about what responsibility have I, and 
•to whom, and then quickly add the 
consequences. That does not mean we have 
to go all the way with Skinner. 

We are accountable and there are 
consequences and we are rewarded. Now 
there is the old business about publish or 
perish and not rewarding teaching. Thus, the 
first answer is there must be rewards, 
carefully tailored to the objectives that we 
Want. To not provide them is a lack of 
system accountability. The other part of this 
is, this may sound romantic, that one of the 
major rewards of teaching is what I would 
call a celebrative experience with which I 
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have been impressed with over the years. My 
original training was as a mechanical 
engineer which did not provide me very 
much in terms of capability. I was a good 
example nf the old system. I am glad 
because I never would have passed if they 
had competency. I had competence only in 
taking tests. For example, I am a machinist, 
too, and I can actually cut a thread on a die. 
The only thing is that the nut will never fit 
on the bolt, but it looks good. It looks like 
the kind of thread that should work. Perhaps 
I can teach you how to do it, however, I 
cannot really do it because no one ever made 
me cut a thread to specification. 

It seems to me one of the rewards is this 
celebrative experience. When I left 
engineering, I went into clinical psychology. 
I was very interested in Freud's notion of 
what he called the oceanic experience, what 
William James calls the mystic experience, 
what Mazlow calls peak experiences and 
what I myself have detected, and am sure 
you have, is the sort of a warm feeling of 
thanksgiving that one is competent and able 
to serve, r think it is possible to talk 
meaningfully about engineering such 
experiences and the key is the overcoming of 
odds. It is interesting, to me at least, that we 
celebrate a birth and not a derm. We 
celebrate a marriage not a divorce. We 
celebrate reaiiy the overcoming of the 
second law of thermodynamics. Life itself 
overcomes the odds. The Catholics celebrate 



the Mass. The word is celebrate, and if you 
are not a believer, the notion is overcoming 
death itcjif through the particular ritual. 

Now, on a very mundane level, it seems to 
me a teacher kind of gets this reward when 
you do overcome the odds and you take the 
high risk student. That does not mean the 
other one is not important. It seems' to me 
one of the exciting things about a profession 
is that it tackles the tough problems, and 
overcomes them. I think the truly 
professional practitioner experiences this. I 
have talked with many people in the Peace 
Corps, who report that when they are 
competent and do something, save a life, or 
build a well and a community thrives where 
it would not have, they get this feeling 
described by Freud and Mazlow and others. 
That is part of the rewards that ought to be 
added. p 

I think it would be a mistake just to talk 
about those things. I would also want/to 
very carefully look at the monetary and 
status rewards. I think, particularly at the 
university level, we really, as the Indians say, 
talk with forked tongue. This is an old tiling. 
We really do not reward teaching. Therefore, 
the- system is not accountable. It lacks 
system accountability and it should be 
addressed as a-major problem. 
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PARTICIPANT OBJECTIVES 



Prior to the Fifth Symposium the participants will 
have read 

- Learning for Mastery by B. S. Bloom 
-Preparing Instructional Objectives by 

R.S.Mager 

- Goal Analysis by R. S. Mager 
-summary and recommendation section of the 

Southern Regional Education Board report 
"Veterinarians for the South," and the 
summary section pertaining to veterinary 
medical education of the committee of the 
National Academy of Science "Report on 
Veterinary Medical Research and Education" 
-selected reprints sent out upon 
preregistration; 

and be able to 

-describe the concept of learning for mastery 
and its implication for veterinary medical 
education 

-list the three criteria for an educational 
objective as described in Preparing 
Instructional Objectives by R-. S. Mager 
-evaluate whether a given objective meets 
. those criteria and if performance criteria 
conform to the concept of learning for 
mastery 

- define a minicourse. 



At the conclLsion of the* Fifth Symposium, the 
participants wiil have: * • 

planned a minicourse which will consist of 
-title 

- objective(s) written for cognitive and/or 
affective domain 

- performance criteria 

-selected media, with justification for choice 

- proposed methods of evaluation; 

■— r 
participated in discussions of 

- minicourses 
-objectives 

-multi-media instruction 

- methods of evaluation 

- National Academy of Science and Southern 
J Regional Education Board reports on 

veterinary medical education; 
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developed a notebook containing 

- copies of the formal presentations 
-available commercial literature on media 

hardware 

-copies of recommendations from the reports 
on Veterinary Medical Education 

- copies of their planned minicourse 
-copies of selected participant-planned 

minicourse 

- notes (personal); 

increased 

-the number of approach reponsesto mastery 
learning and minicourse development; 

and be able to 

-describe and give examples of minicourses 
and their application to veterinary medical 
education 

- describe the advantages and disadvantages of 
"learning for mastery" 

- use a storyboard to plan an enabling 
"instructional program for educational 
objectives 

-list advantages and disadvantages for 
different media and methods of presenting 
instructional materials and describe criteria, 
for choosing 

-operate a videotape recorder and TV camera 
in accordance with acceptable standards 

-list formats for evaluation of a chosen 
competency, along with the :"'/antages of 
each 

-write objectives conformin to criteria 
outlined at the symposium 

- construct a series of evaluative questions that 
can be used to evaluate a chosen competency 

- plan a minicourse. 

Upon ariving home from the Fifth Symposium the 

participants will 

-contact five faculty members and tell them 
about minicourse, mastery learning, and 
multi-media, ^shovying them how they can 
plan a minicourse 
-share with these five faculty members any 
material from the symposium that applies to 
their area, and encourage and assist them in 
developing and producing one minicourse 
during the next year 

- produce two additional minicourses this 
coming year. 
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PROGRAM PARTICIPANTS 



A. R. Allen, M.S. - Director of Medical 
Instruction and Communication, School of 
Veterinary Science and Medicine, Purdue 
University, Lafayette, Indiana 

W. W. Armistead, D.V.M. - Dean, College of 
Veterinary Medicine, Michigan State University, 
East Lansing, Michigan- 

J. H. Block, Ph.D. - Assistant Professor, 
Department of Education, University of 
California, Santa Barbara, California 

R. E. Brown, D.V.M. - Chairman, Departm.ent of 
Veterinary Medicine and Surgery, College of 
Veterinary Medicine, University of Missouri, 
Columbia, Missouri,. 

C. E. Cornelius, D.V.M. - Dean, College of 
Veterinary Medicine, University of Florida, 
Gainesville, Florida 

L. M. Crawford, D.V.M. - Associate Dean, College 
of Veterinary Medicine, University of Georgia, 
Athens, Georgia 

H. Gardner, Ph.D. - Director of Undergraduate 
Studies, Department of Education, Purdue 
University, Lafayette, Indiana 

T. Hale Ham, M.D. -* Director, Division of 
Research in Medical Education, Case Western 
Reserve University, Cleveland, Ohio 

J, W, Harris, Ed.D. - Associate Director, Office of 
Instructional Research and Development, 
University of Georgia, Athens, Georgia 

W. D. Hedges, Ph.D. - Chairman, Department of 
Elementary Education, University of Florida, 
Gainesville, Florida 

L. M, Lessinger, Ph.D. - Calloway Chair of 
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Summary of Replies to 
POST SYMPOSIUM QUESTIONNAIRE 



Rating of Activities: 

1. Conference Objectives 

2. Pre-conference Readimis 

3. Reports on Veterinary 
Medical Education 

4. Minicourse planning 

5. Assembling Notebook 

6. Discussion sessions 

7. Work groups 

8. Luncheon and banquet 
speakers 

9. Media displays 

10. Storyboard planning 

Rating of Topics: 



1. New Horizons for 
Veterinary Medicine 

2. Veterinarians for the South 

3. Minicourses ^> 

4. Performance Objectives 

5. Media - 

- presentation 

- slide tape 
-film 
-T.V. 

- Library Resources 
-Computer 

6. Evaluation 

Mastery learning 

Pre-testing 

Post-testing 

7. Criteria based curriculum 

8. Career Planning 
g. Accountability 

10. Research in Education 



Very Very 
Appropriate Appropriate Inappropriate Inappropriate 

71.8% 28.2% 0% 0% 

44.7 53.5 1.4 0.5 

27.9 67.2 5.0 0.0 

36.4 54.8 7.4 1.4 
38.1 56.3 5.1 0.5 

35.5 60.8 3.2 0.5 
15.0 63.6 20.1 1.3 

26.3 61.7 11.0 1.0 

48.8 50.2 1.0 0.0 

16.9 60.0 20.0 3.1 



Applicability to 
Methods of Presentation Veterinary Medicine 



exc. 


good 


fair 


poor 


exc. 


good 


fair 


poor 


35.4% 


53.1% 


11.5% 


0.0% 


56.8% 


37.4% 


4.7% 


1.19i 


11.8 


51.9 


25.1 


11.2 


36.3 


44.0 


17,3 


2.4 


43.1 


44.5 


11.0 


1.4 


60.6 


35.8 


3.1 


0.5 


50.5 


43.7 


4.9 


1.0 


66.3 


30.5 


3.2 


0.0 


35.2 


52.0 


11.2 


1.7 


53.2 


42.9 


3.2 


0.6 


37.5 


5V.1 


n.4 


0.0 


60.2 


37.3 


2.4 


0.0 


31.1 


49.7 


13.9 


5.3 


56.8 


41.3 


2.6 


0.0 


35.8 


45.7 


14.6 


4.3 


55.1 


35.3 


9.0 


0.6 


26.2 


46.4 


22.0 


5.4 


46.5 


45.3 


6.3 


1.9 


29.9 


51.7 


14.9 


3.4 


39.1 


39.7 


14.7 


6.4 


43.1 


42.7 


10.9 


3.3 


50.0 


40.7 


7.1 


2.2 


17.9 


46.2 


26.7 


9.2 


37.9 


49.2 


10.7 


2.3 


20.3 


50.3 


19.8 


9.6 


43.6 


45.4 


9.1 


1.8 


37.7 


49.3 


11.6 


1.4 


575 


40.9 


6.6 


0.0 


24.6 


46.3 


25.1 


3.9 


41.3 


41.3 


17.3 


0.0 


64.5 


32.2 


3.2 


0.0 


69.1 


28.2 


2.1 


0.5 


38.3 


45.4 


13.7 


2.7 


62.1 


25.9 


11.2 


0.9 
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